🤖 machine learning

VAE with Hyperspherical Coordinates: Improving Anomaly Detection from Hypervolume-Compressed Latent Space

This paper proposes a Variational Autoencoder (VAE) that utilizes hyperspherical coordinates to compress latent vectors toward a specific direction, thereby mitigating the hypervolume expansion problem in high-dimensional spaces and significantly improving both unconditional and conditional anomaly detection performance on complex real-world and benchmark datasets.

Original authors: Alejandro Ascarate, Leo Lebrat, Rodrigo Santa Cruz, Clinton Fookes, Olivier Salvado

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alejandro Ascarate, Leo Lebrat, Rodrigo Santa Cruz, Clinton Fookes, Olivier Salvado

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Empty Room" Effect

Imagine you are trying to teach a robot to recognize what a "normal" galaxy looks like, or what a normal picture of a Martian rock looks like. To do this, the robot uses a Variational Autoencoder (VAE). Think of a VAE as a compression machine. It takes a complex image, squishes it down into a tiny, abstract summary (a "latent vector"), and then tries to un-squish it back into an image.

The goal of anomaly detection is simple: If the robot sees a picture it can't compress well, or if the summary it creates looks weird compared to the summaries of normal pictures, it should sound an alarm. "This is an anomaly!"

But here is the catch: When the summaries are high-dimensional (which they need to be for complex images), they behave strangely.

The paper explains a mathematical phenomenon called "Concentration of Measure." Imagine a giant, invisible balloon (a hypersphere) floating in a room with thousands of dimensions.

The Standard VAE Problem: If you throw darts at this balloon randomly (which is what standard VAEs do), almost all the darts land on the "equator" (the middle band). The "poles" (the top and bottom) are almost empty.
The Consequence: Because the darts are all crowded on the equator, there is so much empty space everywhere else that it becomes impossible to tell the difference between a "normal" dart and a "strange" dart. They are all just floating in a vast, empty ocean of the equator. The robot gets confused because the "normal" data is too spread out.

The Solution: The "Island" Strategy

The authors propose a clever fix: Hyperspherical Coordinates.

Instead of letting the robot describe the image using standard X, Y, Z coordinates (Cartesian), they force the robot to describe the image using angles and a radius (like latitude and longitude on a globe).

The Analogy of the Compass:
Imagine the standard VAE is like a person trying to walk to a specific spot on a giant, flat, foggy field. They can move North, South, East, or West. To get to a specific point, they have to adjust every single direction at once. It's messy, and they tend to end up wandering in the middle of the field (the equator).

The new method (Hyperspherical VAE) is like giving that person a compass and a map of a globe.

The authors tell the robot: "Don't just wander the equator. We want you to walk toward the North Pole."
By changing the math (the "loss function"), they force all the "normal" data summaries to cluster tightly together near the North Pole, forming a dense, compact "island."
Because the North Pole is a tiny, specific spot, the "island" is very crowded.

How It Detects Anomalies

Now, the detection process becomes much easier:

The Setup: The robot trains on "normal" data. All the normal summaries pile up into a tight, dense island near the North Pole.
The Test: When a new image comes in:
- If it's normal, its summary lands right on the island. It's close to all the other normal summaries.
- If it's an anomaly (a weird galaxy or a broken rock), its summary cannot fit on the island. It gets pushed far away into the empty space of the "ocean" (the equator or the southern hemisphere).
The Score: The robot simply measures the distance. "How far is this new point from our crowded island?" If it's far away, it's an anomaly.

The paper claims this is much better than the old way because the "island" is so dense and the "ocean" is so empty that the distance measurement is very clear.

What They Tested

The authors tested this on two very different real-world scenarios:

Mars Rover: They fed the robot images from the Mars Rover. They wanted it to spot unusual rocks or landscapes that didn't look like the "normal" Martian terrain it had seen before.
Galaxy Zoo: They fed it images of galaxies. They wanted it to spot "odd" galaxies that looked different from the typical spiral or elliptical shapes.

They also tested it on standard computer vision benchmarks (like CIFAR-10 and ImageNet), where they tried to spot images that didn't belong to the main categories.

The Results

The paper claims their method works better than existing methods in two main ways:

Unconditional Detection: Finding weird stuff without knowing exactly what the "normal" sub-categories are (e.g., just knowing "this is a galaxy" but not knowing if it's a spiral or elliptical).
Conditional Detection: Finding weird stuff when you do know the sub-categories (e.g., "this is a dog, but it looks like a cat").

In almost every test, their "Island" method (compressing the data toward the pole) found more anomalies and made fewer mistakes than the standard methods, which were stuck wandering the empty equator.

A Note on Speed

The paper admits there is a small cost. Doing the math to convert from "flat coordinates" to "globe coordinates" takes a bit more computing power. It makes the training process about 32% slower than the standard method. However, the authors argue that the massive improvement in accuracy is worth the extra time.

Summary

The paper says: "Standard AI models get lost in high-dimensional space because everything crowds onto the 'equator.' We changed the math to force all the 'normal' data to huddle together at the 'North Pole.' This creates a tight, easy-to-spot island. Anything that doesn't fit on the island is clearly an anomaly, making the AI much better at spotting the weird stuff."

Technical Summary: VAE with Hyperspherical Coordinates for Anomaly Detection

Problem Statement
Variational Autoencoders (VAEs) are widely used for anomaly detection (AD) by encoding data into a lower-dimensional latent space. The standard assumption is that a trained VAE will reconstruct in-distribution (ID) data well but fail on out-of-distribution (OOD) or anomalous data. However, in high-dimensional (HD) latent spaces, standard VAEs suffer from the "concentration of measure" phenomenon. When samples are encoded into a high-dimensional Gaussian prior, they tend to concentrate on the "equators" of the latent hypersphere rather than near the origin or a specific direction. This results in a vast, sparse hypervolume where normal data is distributed uniformly across equatorial bands. Consequently, distinguishing anomalies becomes difficult because the distance between normal samples and anomalies does not exhibit the expected concentration, and the "tails" of the distribution are short, making separation via standard distance metrics ineffective.

Methodology
The authors propose a novel formulation of the VAE latent space using hyperspherical coordinates to mitigate the sparsity and volume issues inherent in HD Cartesian spaces.

Coordinate Transformation: Instead of modeling the latent posterior in Cartesian coordinates ( $\mu, \sigma$ ), the method transforms these into hyperspherical coordinates consisting of a radius $r$ and $n-1$ angular coordinates $\phi_k$ . The transformation is implemented via a vectorized, differentiable PyTorch script to maintain efficiency.
Hypervolume Compression: The core innovation lies in reformulating the Kullback-Leibler Divergence (KLD) loss term.
- In a standard VAE, the prior is a standard Gaussian, which implicitly encourages samples to lie on the equators of the hypersphere due to the geometry of high-dimensional volumes.
- The proposed method sets a specific prior for the angular coordinates ( $\phi_k$ ) such that the mean of the cosine of these angles is driven toward 1 ( $a_{\mu,k} = 1$ ). This forces the latent samples to move away from the equators and toward a specific "north pole" direction on the hypersphere.
- By compressing the latent samples toward a single direction, the method creates a dense, compact "island" of normal data in a region of significantly reduced hypervolume.
Loss Function: The total loss combines the reconstruction error (MSE) with a modified KLD term calculated in hyperspherical coordinates. The KLD term is decomposed into losses for the angular coordinates ( $\phi_k$ ) and the radial coordinate ( $r$ ). The angular loss encourages the samples to align with the target direction, while the radial loss ensures the samples lie on a specific hyperspherical surface (radius $\sqrt{n}$ ).
Anomaly Scoring: After training on normal data, anomalies are detected by encoding a query sample and measuring its distance to the cluster of normal latent means. The authors utilize a simple k-Nearest Neighbors (k-NN) distance metric (with $k=3$ ) on the latent means. Because the normal data is compressed into a dense island, the distance to this island serves as a robust anomaly score.

Key Contributions

Hyperspherical Formulation: The paper introduces a method to convert VAE latent variables from Cartesian to hyperspherical coordinates, effectively disentangling dimensions and allowing for targeted volume compression.
Volume Compression Mechanism: Unlike methods that only compress the radius or a single angle (similar to von Mises-Fisher distributions), this approach compresses all hyperspherical angular coordinates. This allows for a much faster reduction of the latent hypervolume, creating a denser representation of normal data.
Unified Framework: The method is applicable to both fully unconditional-OOD (no subclass labels available) and conditional-OOD (subclass labels available) scenarios. In the conditional case, the coordinate system is re-oriented for each class to form compact clusters for each specific ID class.
Ablation of vMF: The authors demonstrate that their method subsumes von Mises-Fisher (vMF) based approaches as a special case (compressing only one angle) and show that compressing all angles yields superior results.

Experimental Results
The method was evaluated on both complex real-world datasets and standard benchmarks:

Fully Unconditional-OOD:
- Mars Rover Mastcam: Detecting unusual landscape features. The proposed method (Comp.VAE + k-NN) achieved an AUROC of 0.764, outperforming standard VAEs, Autoencoders, and the vMF variant.
- Galaxy Zoo: Detecting unusual galaxies. The method achieved an AUROC of 0.789, again surpassing existing baselines.
Conditional-OOD:
- CIFAR-10 vs. CIFAR-100 (Near-OOD): The method achieved a significant reduction in False Positive Rate at 95% True Positive Rate (FPR95) compared to state-of-the-art methods, achieving an FPR95 of 23.2 (vs. 50.1 for SNN and 58.3 for KNN).
- Imagenette vs. Close ImageNet Classes: In a challenging near-OOD setting, the method achieved the best performance in both FPR95 (78.8) and AUROC (68.3) among the tested configurations, outperforming the vMF-based variant.
Visualization: 3D visualizations of the latent space confirm that while standard VAEs distribute data uniformly over the sphere (equators), the proposed method successfully compresses data into a dense, compact island, making class separation and anomaly detection more intuitive.

Significance and Claims
The paper claims that the primary limitation of standard VAEs in anomaly detection is the geometric sparsity of high-dimensional latent spaces, where normal data occupies the vast equatorial regions of the hypersphere. By reformulating the latent space in hyperspherical coordinates and actively compressing the volume toward a specific direction, the authors demonstrate that:

It is possible to create a "dense island" of normal data, making the separation of anomalies via simple distance metrics (k-NN) significantly more effective.
This approach improves performance in both unconditional and conditional OOD detection tasks without requiring complex contrastive learning or architectural changes beyond the coordinate transformation.
The method provides a theoretical and practical solution to the "curse of dimensionality" in generative models for anomaly detection, offering state-of-the-art results on FPR95 metrics in several benchmarks while maintaining a relatively simple implementation based on standard VAE components.

The authors note that while the coordinate transformation adds computational overhead (approximately 32% slower per epoch for 200 dimensions), the gain in detection performance justifies the cost. They conclude that controlling and reducing the sparsity of the high-dimensional latent manifold is a critical factor for improving not only anomaly detection but potentially other tasks like classification and generation.