🤖 machine learning

ReFP-AD: Rectified Flow Preconditioning for Energy-Based Anomaly Detection

ReFP-AD introduces a geometric reparameterization via optimal transport-coupled rectified flow to stabilize Energy-Based Model training in high-dimensional token spaces, achieving state-of-the-art unified anomaly detection performance on MVTec-AD and VisA datasets.

Original authors: Camile Lendering, Erkut Akdag, Joaquín Figueira, Egor Bondarev

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Camile Lendering, Erkut Akdag, Joaquín Figueira, Egor Bondarev

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to spot a fake painting in a gallery. You've spent years studying thousands of genuine masterpieces, so you know exactly how the brushstrokes, colors, and textures should look. But you've never seen a fake. Your job is to walk into a room full of new paintings and instantly point out the one that doesn't belong. This is the challenge of "anomaly detection" in the world of artificial intelligence. The AI needs to learn what "normal" looks like without ever being shown a "bad" example.

To do this, modern AI often uses "foundation models," which are like super-smart brains trained on the entire internet. These models break images down into tiny pieces called "tokens" and turn them into long lists of numbers (embeddings) that capture the image's meaning. However, these lists of numbers are messy. They are like a tangled ball of yarn where some threads are stretched tight and others are loose, and they are all knotted together in complicated ways. If you try to use these messy lists to mathematically measure how "weird" a new image is, the math gets confused and unstable. It's like trying to walk a tightrope while the rope is constantly twisting under your feet. This paper tackles that specific problem: how to straighten out that tangled rope so the AI can reliably spot the fakes.

The Tangled Rope Problem

The researchers behind this paper, Camile Lendering and their team from Eindhoven University of Technology, noticed a frustrating glitch in how AI detects anomalies. They were using a powerful type of math called Energy-Based Models (EBMs). Think of an EBM as a landscape of hills and valleys. "Normal" images (like a perfect factory part or a healthy leaf) sit comfortably in deep, smooth valleys. "Anomalies" (like a scratch or a defect) are supposed to be high up on the jagged peaks.

The problem is that the AI was trying to explore this landscape using a method called "Langevin dynamics," which is basically a random walk to find the lowest point. But because the data came from those messy, tangled foundation models, the landscape wasn't a smooth valley; it was a twisted, anisotropic (stretched in one direction) nightmare. The random walk would get stuck, bounce off walls, or spiral out of control. It was like trying to roll a marble down a slide that had been crumpled into a ball. Previous attempts to fix this involved squashing the data into a smaller, simpler shape, but that threw away too much detail, making the AI blind to subtle defects.

The Straightening Solution: ReFP-AD

The team proposed a clever new trick called ReFP-AD (Rectified Flow Preconditioning for Anomaly Detection). Instead of trying to force the messy data to behave, they built a "geometric straightener."

Imagine you have a crumpled piece of paper with a drawing on it. If you try to measure the drawing while it's crumpled, your ruler gives you wrong answers. ReFP-AD is like a magical machine that gently uncrumples the paper, stretching it out perfectly flat without tearing the drawing or losing any of the ink. In technical terms, they use a "rectified flow" (a type of mathematical map) to transform the messy, high-dimensional token lists into a clean, well-organized "latent space."

In this new, straightened space, the "hills and valleys" of the anomaly landscape become smooth and predictable. Now, when the AI takes its random walk to find the lowest point (the normal data), it glides smoothly instead of getting stuck. This allows the AI to use the full, rich detail of the original foundation model without having to squish it down into a smaller, less accurate version.

What They Found

The results were impressive. The team tested their method on two major industrial datasets: MVTec-AD (with 15 categories of objects) and VisA (with 12 categories). They used a strict "unified" protocol, meaning they trained just one single AI model to handle all these different objects at once, rather than training a separate model for each.

On the MVTec-AD dataset, ReFP-AD achieved an Image AUROC of 98.6% and a Pixel AUROC of 97.9%.
On the VisA dataset, it scored 97.3% for images and 99.0% for pixels.

To put this in perspective, previous unified models that tried to do this without the "straightening" trick struggled significantly. The paper notes that ReFP-AD outperformed the best previous Energy-Based Model baselines by as much as +10.8% in image detection accuracy.

The researchers ran several experiments to prove their idea was the key. When they removed the "straightening" step and tried to train the AI on the raw, messy data, performance plummeted (dropping to 87.0% on VisA). They also found that because the space was now so well-organized, the AI didn't need to take hundreds of steps to find the answer; it only needed about 20 steps to get it right, whereas before it would have needed many more or failed entirely.

Why It Matters

This paper suggests that the bottleneck in making AI smarter at spotting defects isn't about building bigger brains or more complex models. It's about how we prepare the data for those models to think. By fixing the geometry of the data first, the researchers unlocked the full potential of existing foundation models.

While the method is currently a bit slower than some simpler detectors because it has to do this "straightening" math before making a decision, it proves that we can build a single, unified AI that is incredibly good at spotting defects across many different types of objects. It's a step toward a future where one smart detective can patrol an entire factory, spotting a scratch on a metal gear and a tear in a fabric roll with the same high level of accuracy, all without ever needing to see a broken example to learn what "broken" looks like.

Technical Summary: ReFP-AD

Problem Statement

Visual anomaly detection (VAD) aims to identify deviations from a distribution of normal data without access to anomalous examples during training. In the unified setting, a single model must generalize across diverse object categories, textures, and appearance variations. While foundation models like DINOv2 provide rich, high-dimensional token representations (e.g., 1536-dimensional vectors) that organize visual data into semantically structured manifolds, leveraging these spaces for explicit density estimation remains challenging.

Energy-Based Models (EBMs) offer a principled framework for such tasks by assigning a scalar energy $E(x)$ to inputs, where anomalies correspond to high-energy states. However, training EBMs in high-dimensional token spaces is notoriously unstable. This instability is identified as fundamentally geometric rather than architectural:

Anisotropy and Correlation: Transformer token distributions are highly anisotropic and exhibit strong cross-dimensional correlations.
Metric Mismatch: Standard Langevin dynamics (used for sampling negative examples) assume an isotropic Euclidean metric with isotropic noise. The mismatch between this assumption and the actual token geometry leads to poorly conditioned trajectories.
Sampling Instability: Under finite-step Markov Chain Monte Carlo (MCMC) sampling (e.g., Persistent Contrastive Divergence), this geometric mismatch causes slow mixing, oscillatory "zig-zag" trajectories, and unstable negative phases, often forcing prior methods to compress representations into low-dimensional bottlenecks to stabilize training.

Methodology: ReFP-AD

The authors propose ReFP-AD (Rectified Flow Preconditioning for Anomaly Detection), a framework that decouples representation geometry from density estimation. Instead of modifying the energy model or compressing features, ReFP-AD learns a geometric reparameterization to precondition the sampling space.

1. Unified Token Representation

The method utilizes a frozen DINOv2 ViT backbone to extract patch tokens. To handle scale disparities across categories in a unified setting, tokens are Z-score standardized per category using statistics from normal training data. This serves as a fixed affine pre-processing step before the unified models are applied.

2. Geometric Preconditioning via Optimal Transport Flow

The core innovation is learning a transport map that reshapes the anisotropic token distribution into a well-conditioned, approximately isotropic latent space.

Rectified Flow: A continuous-time transport map is parameterized by a neural network $v_\theta(t, x_t)$ , defined by an Ordinary Differential Equation (ODE).
OT-Coupling: Source tokens $z$ and target isotropic Gaussian variables $u \sim \mathcal{N}(0, \tau^2 I)$ are coupled via an entropic Optimal Transport (OT) coupling, computed using stable log-domain Sinkhorn iterations.
Training Objective: The flow is trained to minimize the matching loss between the network's predicted velocity and the linear interpolation vector $(u-z)$ .
Validation: Checkpoints are selected based on SGLD-fitness diagnostics (conditioning, residual correlation, and tail stability) computed on a projected subspace, ensuring the transport map specifically improves finite-step MCMC behavior without collapsing the manifold topology.

3. Unconstrained EBM with Preconditioned SGLD

Once tokens are transported to the latent space $u$ , an unconstrained Energy-Based Model (a residual MLP) is trained.

Training: The model uses Persistent Contrastive Divergence (PCD). Negative samples are updated using Preconditioned Stochastic Gradient Langevin Dynamics (pSGLD). The preconditioner $M_k$ is computed from a moving average of squared gradients (RMSProp-style), adapting to the local geometry of the learned energy landscape.
Loss Function: The objective minimizes the energy of positive samples while maximizing it for negative samples, with regularization terms for energy magnitude and gradient smoothness.

4. Anomaly Scoring

During inference, anomaly scores are derived directly from the learned energy landscape. The score for a patch is the gradient norm $\|\nabla_u E_\phi(u)\|_2$ , representing the magnitude of the "restoring force" required to move the sample toward the normal manifold. These patch scores are upsampled and smoothed to generate pixel-level anomaly maps.

Key Contributions

Geometric Diagnosis: The paper identifies that the instability of unified token-space EBMs stems from the violation of Euclidean metric assumptions by anisotropic foundation token distributions, rather than model capacity limitations.
Geometric Reparameterization: ReFP-AD introduces a novel strategy using OT-coupled rectified flow to map high-dimensional vision tokens into an isotropic latent space. This enables stable, finite-step MCMC sampling without architectural bottlenecks or dimensionality reduction.
Unified Evaluation: The method is evaluated under a strict unified protocol (single shared flow and EBM for all categories) on MVTec-AD and VisA, demonstrating that explicit density estimation can outperform reconstruction-based and similarity-based baselines when the geometry is properly conditioned.

Experimental Results

The method was evaluated on the MVTec-AD (15 categories) and VisA (12 categories) datasets.

Performance: ReFP-AD achieves state-of-the-art results among density-based detectors:
- MVTec-AD: 98.6% Image AUROC / 97.9% Pixel AUROC.
- VisA: 97.3% Image AUROC / 99.0% Pixel AUROC.
Comparison: It outperforms prior unified EBM baselines (e.g., MPDR) by up to +10.8% in Image AUROC on VisA. It also surpasses leading normalizing flow-based methods (HGAD) on MVTec-AD.
Ablation Insights:
- Necessity of Preconditioning: Removing the rectified flow causes a significant performance drop (e.g., VisA Image AUROC falls from 97.3% to 87.0%), confirming that raw token spaces are unsuitable for finite-step Langevin dynamics.
- Sampling Efficiency: The preconditioned space allows for rapid MCMC mixing. Stable performance is achieved with as few as 20 SGLD steps, whereas standard SGLD in the raw space fails to mix effectively.
- Unified vs. Per-Category: The unified model trails dedicated per-category models by only 0.4–0.6%, demonstrating the feasibility of a single model for heterogeneous data.

Significance and Claims

The paper claims that the primary barrier to scaling Energy-Based Models to foundation representations is sampling geometry, not model capacity. By addressing the geometric mismatch through a learned transport map, ReFP-AD enables:

Stable High-Dimensional Training: EBMs can be trained directly on dense, high-dimensional token representations (1536-D) without collapsing them into low-dimensional bottlenecks.
Preservation of Semantics: Unlike methods that compress features, ReFP-AD preserves the rich, structured token information necessary for accurate localization and multi-class modeling.
General Principle: The work suggests a general paradigm for energy-based modeling: first condition the representation geometry, then learn the energy.

The authors acknowledge that while the method improves detection robustness, the requirement for ODE integration (10-step RK4 solver) and gradient evaluation increases inference latency compared to feed-forward or k-NN based detectors, making it better suited for offline inspection or scenarios where unified modeling outweighs strict real-time constraints.