🤖 machine learning

Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection

This paper introduces K-DSM, a kurtosis-guided denoising score matching method that adaptively scales noise per feature to achieve state-of-the-art tabular anomaly detection in both semi-supervised and fully unsupervised settings without requiring complex multi-scale training or extensive hyperparameter tuning.

Original authors: Victor Livernoche, Jie Zan, Reihaneh Rabbany

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Victor Livernoche, Jie Zan, Reihaneh Rabbany

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a very busy, chaotic train station. Your job is to spot the one person who doesn't belong—maybe they are wearing a tuxedo in the middle of a summer rush, or carrying a giant, invisible suitcase.

This is the job of Anomaly Detection. For a long time, computers have tried to do this by learning what "normal" looks like and then flagging anything that looks weird.

This paper introduces a new, smarter way to teach the computer how to spot these weirdos, specifically for tabular data (think of spreadsheets with rows and columns, like bank transactions or medical records).

Here is the story of their new method, K-DSM, broken down into simple parts.

1. The Problem: The "Goldilocks" Noise Dilemma

The method they use is called Denoising Score Matching (DSM). To understand it, imagine you take a clear photo of a normal person and then spray it with a little bit of fog (noise). You then train a computer to "de-fog" the image and guess where the original person was standing.

The Score: If the computer has to push the "foggy" point really hard to get it back to a normal spot, that means the point was probably weird to begin with. That "push" is the anomaly signal.
The Dilemma: How much fog (noise) should you spray?
- Too little fog: The computer only learns about the crowded center of the station. It misses the weirdos standing in the empty corners.
- Too much fog: The whole station gets so blurry that the computer can't tell the difference between a normal person and a weirdo. Everything looks the same.

Usually, researchers try to solve this by using many different amounts of fog at once (multi-scale). But this is slow, expensive, and complicated.

2. The Solution: The "Tailored Fog" (Kurtosis)

The authors realized that not all features in a spreadsheet are the same. Some columns are like a calm lake (data is spread out evenly), while others are like a volcano (data is piled up in one spot with a few wild outliers far away).

They introduced a concept called Kurtosis. In simple terms, kurtosis measures how "spiky" or "heavy-tailed" a distribution is.

Low Kurtosis (Flat): The data is spread out. You only need a tiny bit of fog to test it.
High Kurtosis (Spiky/Heavy Tail): The data is clumped together with wild outliers. You need a lot of fog to reach the edges and teach the computer what's normal there.

The Analogy:
Imagine you are teaching a dog to find a ball.

If the ball is in a wide, open field (Low Kurtosis), you only need to throw the ball a few feet to train the dog.
If the ball is hidden in a deep, narrow cave with a long tunnel (High Kurtosis), you have to throw the ball way down the tunnel to train the dog properly.

K-DSM automatically calculates how "spiky" each column of your data is and applies the perfect amount of fog to that specific column. It doesn't use one fog level for everyone; it customizes the fog for every single feature.

3. The "Clean-Up" Trick (EMA-Teacher)

There is a catch: What if your training data (the "normal" photos) already has some weirdos mixed in? (This is called a "contaminated" setting). If you train on them, the computer learns that "weird" is actually "normal."

To fix this, the authors added a Teacher Filter.

Imagine you have a student (the main AI) and a teacher (a slightly older, slower version of the AI).
Before the student tries to learn from a batch of data, the teacher takes a quick look.
If the teacher sees a data point that looks very weird (high score), it says, "Hey, this looks suspicious. Let's skip this one for now."
The student then only learns from the "clean" data the teacher approved.

This prevents the student from accidentally learning that the anomalies are actually normal.

4. The Results: Faster and Smarter

The paper tested this on 57 different real-world datasets (like fraud detection and medical records).

Speed: Because K-DSM uses just one fog level per feature (instead of many complex levels), it is incredibly fast. It's like taking a single, perfect photo instead of taking 100 blurry ones and trying to stitch them together.
Accuracy: It beat almost every other method on the list, including the complex, multi-fog methods.
Simplicity: It requires very little "tuning" by humans. The math (based on the shape of the data) does the work for you.

Summary

The paper argues that you don't need a complicated, multi-layered system to find anomalies in spreadsheets. Instead, you just need to:

Look at the shape of your data.
Give each column the exact amount of "noise" it needs to learn properly.
Use a simple filter to ignore bad data during training.

This makes the system faster, more accurate, and easier to use than the previous state-of-the-art methods.

Technical Summary: Kurtosis-Guided Denoising Score Matching for Tabular Anomaly Detection

Problem Statement

Anomaly detection in tabular data requires identifying observations that deviate from the expected distribution. While Denoising Score Matching (DSM) offers a powerful framework for modeling high-dimensional distributions by learning the score function (the gradient of the log-density), it faces a critical practical challenge: noise scale selection.

Standard DSM relies on a single, fixed noise scale ( $\sigma$ ) applied globally to all features. This creates a trade-off:

Too little noise: The learned score estimates are unstable in sparse regions, failing to capture the structure where anomalies typically reside.
Too much noise: The perturbation erases local structure, smoothing out the distinctions between normal and anomalous samples and weakening sensitivity.

Existing solutions, such as Multi-Scale Score Matching (MSM) or Noise Conditional Score Networks (NCSNs), attempt to resolve this by training models across multiple noise scales. However, these approaches introduce significant computational overhead, require complex inference-time noise selection, and often fail to outperform simpler baselines when architectures are not carefully tuned. Furthermore, in fully unsupervised (contaminated) settings where training data contains anomalies, standard DSM fails because the denoising objective explicitly teaches the model to assign low score norms to the contaminants.

Methodology

The authors propose K-DSM (Kurtosis-Guided Denoising Score Matching), a method that replaces the global noise scale with a data-adaptive, per-feature noise scaling strategy.

1. Per-Feature Perturbation

Instead of applying a global $\sigma$ , K-DSM employs a diagonal noise covariance matrix $\Sigma = \text{diag}(\sigma_1^2, \dots, \sigma_d^2)$ . The objective function is modified to predict the unit-variance noise $\varepsilon$ rather than the raw score, decoupling the gradient contribution from the noise scale:
$\mathcal{L}_{\text{per-feature}}^{\text{DSM}}(\theta) = \frac{1}{2} \mathbb{E}_{x \sim p_{\text{data}}} \mathbb{E}_{\tilde{x} \sim \mathcal{N}(x, \Sigma)} \left[ \| s_\theta(\tilde{x}) + \Sigma^{-1/2}(\tilde{x} - x) \|_2^2 \right]$
This ensures that features with smaller noise scales do not dominate the loss function.

2. Kurtosis-Guided Noise Scaling

The core innovation is determining $\sigma_j$ for each feature $j$ based on the statistical shape of its marginal distribution. The authors utilize kurtosis (the fourth standardized moment) as a proxy for tail extremity.

Rationale: Light-tailed or flat distributions (low kurtosis) already cover their effective support and require smaller perturbations. Heavy-tailed or sharply concentrated distributions (high kurtosis) leave surrounding space underrepresented and require larger perturbations to expose the network to low-density regions.
Histogram Rearrangement: To ensure kurtosis reflects tail weight rather than artifacts like skewness or bimodality, the authors apply a symmetric decreasing histogram rearrangement before computing kurtosis. This step symmetrizes the empirical distribution about its dominant mode, isolating the rate of decay from peak to tails.
The Scaling Rule: The noise scale is computed via an affine rule derived from the Cornish–Fisher expansion:
$\sigma_j = \sigma_{\text{base}} \cdot (1 + c \cdot (\kappa_j^* - 3))$
where $\kappa_j^*$ is the kurtosis of the rearranged $j$ -th feature, $\sigma_{\text{base}}$ is a global hyperparameter, and $c$ is a sensitivity coefficient (empirically set to 0.33). The result is clipped to a valid range $[\sigma_{\min}, \sigma_{\max}]$ .

3. Theoretical Justification

The paper proves that for symmetric unimodal distributions, the tail radius (the distance a training point must be displaced to reach a specific tail probability) is strictly increasing in kurtosis. Consequently, the affine rule in Eq. (5) is shown to be the unique first-order optimal noise scale for tail coverage, providing a principled alternative to heuristic tuning.

4. Unsupervised Extension (EMA-Teacher Filtering)

To address the "contaminated" setting where training data contains anomalies, the authors introduce a lightweight filtering mechanism. They maintain an Exponential Moving Average (EMA) of the online model's weights (a "teacher"). During training, samples with a high score norm according to the teacher are identified as probable contaminants and dropped from the gradient update. This prevents the model from learning to assign low anomaly scores to the contaminants themselves.

Key Contributions

Re-evaluation of Single-Scale DSM: The authors demonstrate that a single-scale DSM model, when equipped with a modern MLP architecture (ResNet-like), already outperforms multi-scale methods (MSM) and other deep baselines (DTE, SLAD) on tabular benchmarks. This challenges the prevailing assumption that multi-scale conditioning is strictly necessary for effective score-based anomaly detection.
K-DSM with Theoretical Guarantees: The proposal of a per-feature noise scaling rule based on marginal kurtosis, supported by a proof that the affine mapping is the unique first-order optimal solution for tail coverage.
State-of-the-Art Performance: K-DSM achieves the best mean AUC-PR (0.627) on the ADBench (57 datasets) in the semi-supervised setting, outperforming the previous best baseline (0.594).
Robust Unsupervised Detection: By combining K-DSM with the EMA-teacher filtering rule, the method achieves state-of-the-art performance (0.343 mean AUC-PR) in the fully unsupervised (contaminated) setting, significantly recovering the performance lost due to data contamination.
Efficiency: Unlike multi-scale approaches that require inference at multiple noise levels, K-DSM maintains the inference speed of a single forward pass, making it significantly faster than MSM and DDAE.

Experimental Results

Semi-Supervised Setting (Clean Training Data): On ADBench, K-DSM achieves a mean AUC-PR of 0.627, surpassing the runner-up baseline (0.594) and the single-scale DSM baseline (0.584). The method is also robust to the choice of the base noise scale $\sigma_{\text{base}}$ , whereas standard DSM is highly sensitive to this hyperparameter.
Unsupervised Setting (Contaminated Training Data): Standard DSM and K-DSM collapse in this setting (dropping to ~0.188 AUC-PR) because they learn to treat anomalies as normal. The EMA-teacher extension restores performance, lifting K-DSM-EMA to 0.343 AUC-PR, the best result among 14 compared methods.
Image Benchmarks: On MVTec-AD and VisA (using DINOv3 embeddings), K-DSM is competitive but does not dominate. The authors note this is expected because standardized deep vision features are approximately Gaussian per coordinate, rendering the kurtosis-based scaling ineffective (as $\kappa \approx 3$ ). This delineates the regime of validity for the method: it excels where marginal tail heterogeneity exists.
Inference Speed: K-DSM is the fastest deep learning method at inference, requiring only a single forward pass (~0.02–0.03s per dataset), compared to seconds for multi-scale or diffusion-based methods.

Significance and Claims

The paper argues that the long-standing challenge of noise scale selection in DSM can be addressed without resorting to computationally expensive multi-scale models. By adapting the noise level to the intrinsic geometry of each feature's distribution, K-DSM improves coverage of low-density regions while preserving sensitivity in high-density areas.

The authors claim that careful, data-adaptive noise scaling combined with a contamination-robust training recipe offers a practical, scalable, and state-of-the-art alternative to heavier ensemble or multi-scale approaches for real-world tabular anomaly detection. The method reduces reliance on hyperparameter tuning and validation sets, making it suitable for scenarios where anomalies are unknown and data is scarce or contaminated.

Limitations Acknowledged:

The method's advantage diminishes when features are standardized to near-Gaussian marginals (e.g., deep vision embeddings), as the kurtosis rule collapses to a single scale.
In the unsupervised setting, the EMA-teacher filter relies on a fixed percentile cutoff, implicitly assuming the contamination rate is below a certain threshold (e.g., <20%). Performance degrades if contamination rates are extremely high.