🤖 machine learning

Exploiting Local Flatness for Efficient Out-of-Distribution Detection

This paper introduces Fold, a lightweight out-of-distribution detector that exploits the observation that OOD inputs exhibit larger Hessian curvature than in-distribution data, utilizing feature Hessian and partial normalization to achieve state-of-the-art performance with minimal computational overhead.

Original authors: Seonghwan Park, Hyunji Jung, Dongyeop Lee, Namhoon Lee

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Seonghwan Park, Hyunji Jung, Dongyeop Lee, Namhoon Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot that has spent years studying a specific library of books (let's say, books about cats and dogs). This robot is an expert at identifying cats and dogs. However, one day, someone hands it a picture of a toaster or a cloud.

Because the robot has never seen a toaster, it doesn't know what to do. But here's the problem: the robot is overconfident. It might look at the toaster and say, "That is definitely a very strange cat!" with 100% certainty. This is dangerous in the real world. We need a way for the robot to say, "Wait, I don't know what this is," instead of guessing wrong.

This paper introduces a new, fast, and clever way to catch these "unknown" items (called Out-of-Distribution or OOD data) without needing to retrain the robot or slow it down.

Here is the breakdown of their solution, FOLD, using simple analogies:

1. The Core Idea: The "Wobbly Table" vs. The "Stable Table"

The researchers discovered a hidden geometric property in how the robot "thinks."

The Known Stuff (In-Distribution): When the robot looks at a cat or a dog, it feels very stable. Imagine the robot's brain is a table. When it sees a cat, the table is perfectly flat and solid. If you push it slightly, it doesn't wobble.
The Unknown Stuff (Out-of-Distribution): When the robot sees a toaster, the table becomes wobbly and sharp. It's like standing on a jagged, uneven rock. If you push it even a tiny bit, the table shakes violently.

The paper proves that for unknown items, the "wobble" (mathematically called curvature) is much stronger than for known items. The more different the item is from what the robot learned, the more the table wobbles.

2. The Problem with Previous Methods

Before this, scientists tried to measure this "wobble" by checking the entire robot's brain (all its billions of connections).

The Analogy: Imagine trying to measure the stability of a skyscraper by checking every single brick, bolt, and wire inside it. It would take forever and require a massive team. This was too slow for real-time use.

3. The Solution: FOLD (The "Shortcut")

The authors created a method called FOLD that measures the wobble without checking the whole building.

The Feature Hessian (The Shortcut): Instead of checking the whole brain, they look at the "feature" layer—the part of the brain that recognizes shapes and textures. They realized they could calculate the wobble just by looking at this specific layer.
The Analogy: Instead of inspecting every brick in the skyscraper, they just check the foundation. If the foundation is shaking, the whole building is unstable. This is incredibly fast, taking about the same time as just looking at the picture once.

4. The "Volume Knob" Trick (Partial Normalization)

There was a catch. Sometimes, the robot gets so excited about an image that the "volume" of its signal gets too loud, which hides the wobble. It's like trying to hear a faint creak in a room where someone is blasting heavy metal music.

The Fix: They introduced a "volume knob" (called Partial Normalization).
- For simple pictures (like a clear photo of a cat), they turn the volume down almost all the way to hear the subtle wobbles.
- For complex pictures (like a busy city scene), they only turn the volume down a little bit, because the "loudness" itself might contain useful clues.
Why it matters: This ensures the robot can hear the "wobble" clearly, no matter how complex the image is.

5. The Self-Adjusting Tuner (AUTOFOLD)

Usually, to set the perfect "volume knob," you need a test set of unknown items (like a box of toasters and clouds) to practice on. But in the real world, you often don't have a box of unknown items waiting for you.

The Magic Trick: They created AUTOFOLD. This system creates "fake unknowns" right there in the moment.
The Analogy: Imagine the robot is looking at a cat. AUTOFOLD says, "Okay, let's pretend this cat is actually a toaster." It tricks the robot by hiding the "cat" answer and forcing it to guess. This creates a fake "wobble" that the robot can use to calibrate its volume knob automatically.
The Result: The robot tunes itself perfectly without needing any external data or extra training.

6. The Results: Fast and Accurate

The paper tested this on huge datasets (thousands of images).

Performance: FOLD caught more "unknowns" than previous methods. It reduced the number of times the robot confidently guessed the wrong thing by a significant margin.
Speed: It is as fast as a standard "forward pass" (just looking at the image once). It doesn't slow the robot down at all.
Efficiency: It sits in the "sweet spot" of the graph: high accuracy with low cost.

Summary

The paper presents FOLD, a method that detects when a robot is confused by an unknown object by measuring how "wobbly" its internal logic is.

Unknown items make the robot's logic wobble more.
FOLD measures this wobble quickly by looking at a specific layer of the brain.
It uses a smart volume knob to ensure the wobble is always audible.
It tunes itself by creating fake unknowns, so it works even if you have no test data.

This allows AI systems to be safer and more reliable in the real world, knowing when to say "I don't know" instead of guessing confidently.

Technical Summary: Exploiting Local Flatness for Efficient Out-of-Distribution Detection

Problem Statement

Reliable machine learning deployment requires the ability to detect out-of-distribution (OOD) data—samples drawn from distributions disjoint from the training set. Modern deep neural networks often produce overconfident, erroneous predictions on such inputs. While post-hoc detection methods (operating on pre-trained models without retraining) are attractive for their efficiency, existing curvature-based approaches face significant limitations. Methods utilizing the loss landscape curvature (e.g., via Laplace approximation) to estimate model uncertainty are computationally prohibitive for large networks due to the quadratic scaling of the Hessian matrix. Furthermore, these methods rely on implicit assumptions regarding the difference in landscape flatness between in-distribution (ID) and OOD data, which have not been systematically investigated.

Methodology

1. Empirical Observations on Curvature Discrepancy

The authors first provide a systematic empirical analysis of the loss landscape curvature induced by ID and OOD samples. Using Hutchinson's method to estimate Hessian statistics (largest eigenvalue $\lambda_{max}$ and trace $tr(H)$) without constructing the full matrix, they observe:

ID vs. OOD Curvature: ID samples consistently exhibit lower curvature (smaller $\lambda_{max}$ and trace) compared to OOD samples.
Shift Severity: The curvature gap widens as the distributional shift becomes more severe, moving from near-OOD to far-OOD regimes.
Sample-Level Separability: The distribution of per-sample Hessian traces shows clear separation, with ID samples concentrated at lower values and OOD samples shifting toward higher values with broader distributions.

2. FOLD: Flatness-Modulated OOD Detection

Motivated by the observation that ID samples lie in flatter regions of the loss landscape, the paper proposes FOLD, a lightweight detector that avoids the prohibitive cost of parameter-space curvature approximation.

Feature Hessian: Instead of computing the full parameter Hessian, FOLD transports logit-space curvature back to the latent feature space. Given a network decomposed into a feature encoder $h(\cdot)$ and a linear classifier $g(\cdot)$ , the feature Hessian is derived via the second-order chain rule:
$\nabla_{\mathbf{h}}^2 \mathcal{L} = (\nabla_{\mathbf{h}} g) (\nabla_{\mathbf{z}}^2 \mathcal{L}) (\nabla_{\mathbf{h}} g)^{\top}$
This operator projects predictive uncertainty onto the classifier-induced feature manifold, capturing geometric structures critical for OOD detection.
Partial Feature Normalization: To address the issue where large feature magnitudes cause the softmax to saturate (flattening the loss landscape and vanishing the Hessian), FOLD introduces partial feature normalization:
$\tilde{\mathbf{h}} = \frac{\mathbf{h}}{\|\mathbf{h}\|^{\alpha}}, \quad 0 < \alpha \leq 1$
The detection score is defined as the trace of the Hessian in this normalized space: $S_{fold}(\mathbf{x}) = \text{tr}(\nabla_{\tilde{\mathbf{h}}}^2 \mathcal{L})$ $S_{f o l d} (x) = tr (\nabla_{\tilde{h}}^{2} L)$ .
- Dataset Dependency: The optimal $\alpha$ varies by dataset complexity. Simple datasets (e.g., CIFAR) benefit from near-full normalization ( $\alpha \approx 1$ ), while complex datasets (e.g., ImageNet) require milder suppression ( $\alpha \approx 0.2$ ) to preserve informative magnitude cues.

3. AUTOFOLD: Self-Supervised Calibration

To eliminate the need for external OOD validation sets to tune $\alpha$ , the authors introduce AUTOFOLD. This self-supervised scheme:

Generates pseudo-OOD samples by masking the dominant logit of ID samples (setting the logit of the true class to $-\infty$ ), simulating an unknown-class scenario.
Automatically selects the optimal $\alpha^*$ by maximizing the AUROC between the scores of the original ID validation samples and their logit-masked counterparts.
Operates entirely at inference time with negligible overhead, requiring no retraining or external data.

Key Contributions

Systematic Curvature Analysis: The first systematic investigation confirming that OOD inputs exhibit larger Hessian curvature than ID data, with the discrepancy increasing under stronger distributional shifts.
Efficient Detection Algorithm (FOLD): A lightweight method leveraging the feature Hessian and partial feature normalization to enhance ID-OOD separability without the overhead of Bayesian curvature-based methods.
Automatic Hyperparameter Tuning (AUTOFOLD): A self-supervised strategy using ID logit masking to calibrate normalization parameters without external OOD data.
Theoretical and Empirical Validation: Theoretical analysis (under simplified binary classification assumptions) supports the claim that the expected trace of the feature Hessian is larger for OOD data. Extensive experiments validate these findings across diverse architectures and datasets.

Experimental Results

The authors evaluated FOLD and AUTOFOLD on standard OOD benchmarks (CIFAR-10, CIFAR-100, ImageNet-200, ImageNet-1K) against a comprehensive set of post-hoc baselines (e.g., MSP, ODIN, ReAct, VIM, KNN).

Performance: FOLD and its variants consistently outperform prior methods.
- FOLD-R (FOLD combined with ReAct's activation thresholding) achieved the best average performance, improving the average AUROC by 1.63% and reducing FPR95 by 2.30% compared to strong baselines.
- AUTOFOLD variants achieved competitive performance to fixed- $\alpha$ FOLD, demonstrating that automatic calibration is effective without manual tuning.
Efficiency: FOLD maintains computational efficiency comparable to a standard forward pass. Unlike methods requiring expensive setup (e.g., KNN, MDS) or inference (e.g., RMDS), FOLD incurs near-minimal overhead.
Generalization: The method demonstrates stable performance across diverse backbone architectures (RegNet, DenseNet, WRN, ResNeXt), indicating that the geometric advantages of curvature-based scoring are architecture-agnostic.
Ablation Studies:
- Normalization: Partial normalization was found to be superior to full normalization on complex datasets (ImageNet), where full normalization discarded informative magnitude cues, collapsing the spectral gap between ID and OOD.
- Feature Hessian: Using the feature Hessian yielded higher AUROC than using the logit Hessian alone, particularly on complex domains, confirming the value of incorporating classifier-induced geometry.

Significance and Claims

The paper claims to provide a principled and practical solution for robust real-world OOD detection. By exploiting the structural property that ID data lies in flatter loss regions than OOD data, FOLD bridges the gap between the theoretical promise of curvature-based uncertainty estimation and the practical constraints of computational efficiency.

The authors emphasize that their approach:

Avoids the "prohibitive computational overhead" typical of Bayesian curvature approximations.
Removes the dependency on external OOD validation sets for hyperparameter tuning via AUTOFOLD.
Offers a "plug-and-play" component (partial normalization) that enhances existing detection frameworks.
Provides a theoretical foundation (Theorem 1) linking feature Hessian traces to distributional shifts, offering an intuitive explanation for the observed empirical success.

The work concludes that FOLD achieves a favorable balance between detection accuracy and computational efficiency, making it suitable for deployment in scenarios where OOD data is unavailable for validation and real-time inference is required.