🤖 machine learning

FloDR: An invertible dimensionality reduction method based on a normalising flow

FloDR is an invertible dimensionality reduction method based on normalizing flows that preserves high-dimensional information by retaining unused coordinates, enabling the generation of diagnostic visualizations like conditional spread and hidden contrast to quantify the reliability and information loss of 2D embeddings.

Original authors: Abdallah Baraka, Daniel Probst

Published 2026-07-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abdallah Baraka, Daniel Probst

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to fit a giant, three-dimensional sculpture into a flat, two-dimensional photograph. No matter how you turn the sculpture, some parts will overlap, some distances will stretch, and some details will get squished out of existence. This is the daily struggle of data scientists working with "high-dimensional data." Whether it's millions of pixels in an image, thousands of chemical reactions, or the complex genetic code of a single cell, this information lives in a world with hundreds or even thousands of directions. To make sense of it, we use "dimensionality reduction" to squash it down onto a flat map we can actually see.

But here's the catch: when you squash a 3D object into 2D, you lose information. Traditional tools like t-SNE or UMAP are like skilled photographers who take a great picture, but they throw away the "depth" data. Once the photo is taken, you can't tell if two dots that look close together are actually neighbors in the real world, or if they just happened to land on top of each other by accident. You also can't tell how much of the original object's shape is hidden behind the flat surface. Scientists have long worried that people are reading too much into these flat maps, guessing at distances and groupings that the map simply cannot support. The big question has been: Can we make a map that looks good, preserves the big picture, and also tells us exactly what we don't know?

Enter FloDR, a new method that acts less like a camera and more like a magical, reversible projector. Instead of just taking a snapshot and throwing away the rest of the data, FloDR uses a special kind of mathematical engine called a "normalizing flow." Think of this engine as a stretchy, transparent sheet that can be folded and twisted to flatten the data, but with a superpower: it never tears or loses the fabric. It keeps the "depth" information safe in a hidden pocket.

The paper introduces FloDR as a way to create these 2D maps while keeping a secret backup of all the lost information. When you look at the map, you see the first two coordinates, just like a normal picture. But underneath the hood, FloDR remembers the other $D-2$ dimensions (the "residuals"). Because the math is perfectly reversible, the authors can run the map backward to ask questions like, "If I pick this spot on the map, how much of the original data is still a mystery?" or "How much information about a specific label (like a cell type) did we accidentally throw away?"

The researchers found that FloDR is a strong competitor to the most popular tools out there. On four different benchmark datasets—including images of handwritten digits and chemical reaction data—FloDR managed to keep the local neighborhoods (the small groups of similar things) almost as well as the best existing methods, while doing a much better job at preserving the global structure (how the big groups relate to each other). In fact, on the datasets tested, FloDR was better than UMAP at balancing both local and global structure simultaneously.

However, the paper is careful not to claim FloDR is perfect at everything. It explicitly notes that if your only goal is to keep tiny local neighborhoods perfect, another tool called openTSNE is still slightly better. FloDR also admits that while it can project new, unseen data points onto the map in a single instant, those points might not land in the exact right local neighborhood unless you run a quick extra optimization step. But for the most part, FloDR offers a "best of both worlds" scenario: a map that looks right, feels right, and comes with a built-in "truth meter."

This truth meter is the paper's most playful and powerful feature. FloDR generates two special "diagnostic fields" that you can paint over the map:

Conditional Spread: Imagine a fog that gets thicker in certain areas. This fog tells you how much the original data varies at that spot. If the fog is thick, it means the map has squished together things that were actually very different in the real world.
Hidden Contrast: This is like a "secret decoder ring." It tells you how much information about a specific label (like "is this a cancer cell?") is hidden in the parts of the data you can't see. If the hidden contrast is high in a spot, it means the 2D map is hiding a big difference between the groups sitting there.

Crucially, the authors didn't just guess these values; they tested them rigorously. They split their data, trained the map on one part, and then tried to predict the other part to see if the diagnostic fields were actually telling the truth. They found that for most datasets, these fields passed the test with high confidence. For example, on a dataset of human fetal bone marrow cells, the hidden contrast was certified with a p-value of 0.01, meaning the signal was very strong and not just random noise.

In the end, FloDR doesn't just give you a pretty picture; it gives you a picture with a manual that explains its own limitations. It shows you where the map is trustworthy and where it's just an illusion. By keeping the "residual" data safe and using a reversible mathematical trick, it allows scientists to see not just the shape of their data, but also the shadows it casts, ensuring they don't draw conclusions the projection was never able to support.

Technical Summary: FloDR – An Invertible Dimensionality Reduction Method Based on a Normalising Flow

1. Problem Statement

High-dimensional data is frequently visualized using two-dimensional embeddings (e.g., t-SNE, UMAP). However, these methods often force readers to interpret the resulting maps beyond their statistical support. Distances between clusters, the significance of empty spaces, and the amount of structure hidden at specific points are generally invisible because the optimization processes of standard methods discard the information required to support such interpretations. Furthermore, existing methods face a trade-off: neighbor embeddings preserve local structure but distort global geometry, while linear methods (like PCA) preserve global structure but blur local neighborhoods. Additionally, parametric approaches that allow for out-of-sample embedding often rely on separate, approximate encoder-decoder pairs that are not true inverses, mixing model errors into explanations.

2. Methodology

FloDR (Flow-based Dimensionality Reduction) addresses these issues by embedding data through an invertible normalising flow (a bijection $f_\theta: \mathbb{R}^D \to \mathbb{R}^D$ ) rather than a non-invertible projection.

Architecture

Mapping: The core is a composition of $L=4$ affine coupling layers (RealNVP style) without mixing layers. The input is first whitened and rotated via a fixed linear transformation to initialize the layout on the first two principal components.
Embedding vs. Residual: The model outputs $D$ coordinates. The first two coordinates ( $y \in \mathbb{R}^2$ ) serve as the 2D embedding. The remaining $D-2$ coordinates ( $r \in \mathbb{R}^{D-2}$ ) are retained as a residual rather than discarded.
Invertibility: Because the map is a bijection, any pair $(y, r)$ maps back to an input $x$ (up to floating-point round-off) via the exact inverse $f^{-1}_\theta$ . This allows for exact density estimation via the change of variables formula.

Training Objective

The layout is trained using a composite loss function applied only to the 2D coordinates $y$ :

Local Structure: Follows UMAP by constructing a fuzzy $k$ -nearest-neighbor graph ( $k=15$ ) with attraction and repulsion terms.
Global Structure:
- A bounded mid-range attraction term to counteract repulsion.
- Ordinal Term: A non-metric MDS term that optimizes the sign of distance comparisons between pairs of points in the input space versus the embedding space. This term is controlled by a user-adjustable weight $w$ .
Density Term: A negative log-likelihood term is optionally ramped in during training to prevent the inverse map from over-stretching the residual coordinates.

Out-of-Sample Placement

New data points are embedded via a single forward pass ( $y' = f_\theta(x')_{1:2}$ ). While this preserves global structure, it may fail to preserve local neighborhoods. To address this, FloDR offers an optional placement step (similar to UMAP's transform) where the position of a new point is optimized for 200 steps against the attraction term of the training neighbors, while the trained map remains frozen.

3. Key Contributions: Certified Diagnostics

The primary innovation of FloDR is the ability to generate diagnostic fields that are properties of the distribution of inputs projected to a given 2D position, rather than artifacts of an auxiliary model. These fields are derived from the exact inverse and the residual distribution $p(r|y)$ .

Two Diagnostic Fields

Conditional Spread ( $\sigma(y)$ ):
- Definition: $\sigma(y) = \sqrt{\text{trace}(\text{Cov}(x | y))}$ .
- Meaning: Measures how much of the original input data remains undetermined at a specific embedding position $y$ . High values indicate that the layout has merged inputs that were distinct in the original high-dimensional space.
- Estimation: Computed by drawing residuals from the fitted $p(r|y)$ and mapping them back through the exact inverse.
Hidden Contrast ( $h(y)$ ):
- Definition: $h(y) = I(G; R | Y=y)$ , the mutual information between a labelled contrast $G$ (e.g., cell type) and the residual $R$ , given the embedding position $Y$ .
- Meaning: Measures how much information about a labelled group is discarded by the two plotted coordinates. High values indicate that points drawn close together belong to different groups that the residual can still distinguish.
- Estimation: Calculated using capacity-matched classifiers to estimate $p(G|y)$ and $p(G|y, r)$ .

Certification Process

To prevent over-interpretation, every field undergoes a held-out certification process:

Fields are fitted on a training set and validated on a held-out set.
Validation involves binning the embedding space and regressing the field's prediction against the actual held-out measurement.
Criteria: A field is "certified" if it tracks the validation target with a slope $b \in [0.7, 1.3]$ and $R^2 \geq 0.6$ .
Null Testing: For hidden contrast, the field is compared against a null hypothesis where labels are shuffled within bins.
Outcome: Fields that fail these tests are explicitly marked as "refused" in the visualization, ensuring users do not trust unreliable diagnostics.

4. Results

The authors evaluated FloDR on four benchmark datasets (MNIST, Fashion-MNIST, Paul15 myeloid progenitors, Schneider reaction data) and three single-cell atlases (human fetal bone marrow, mouse cerebellum, Arabidopsis seed).

Global vs. Local Trade-off: FloDR achieves a near-vertical trade-off curve. By tuning the ordinal weight $w$ , global structure preservation (measured by Spearman correlation of pairwise distances, CPD) increases substantially (e.g., from 0.337 to 0.562 on MNIST) with negligible loss in local neighborhood recall.
Comparison to Baselines:
- vs. UMAP: FloDR outperforms UMAP on both global and local axes across all four benchmarks.
- vs. openTSNE: openTSNE retains slightly better local structure but at a significant cost to global structure.
- vs. PCA: PCA preserves global structure well but fails at local neighborhoods. FloDR matches or exceeds PCA's global performance while maintaining local structure.
- vs. SQuadMDS: While SQuadMDS achieves higher global CPD on single-cell atlases, it retains almost no local neighborhood information (recall $\approx 0$ ).
Out-of-Sample Performance: The forward pass of FloDR preserves global structure effectively. When the optional placement step is used, local fidelity is recovered to levels comparable to openTSNE on benchmarks.
Efficiency: FloDR is computationally efficient, completing embeddings of $10^6$ points in 222 seconds on a GPU, faster than UMAP (403s) and t-SNE (273s).

5. Significance and Claims

The paper claims that FloDR represents a shift in how dimensionality reduction is interpreted:

Single-Object Consistency: Unlike methods that use separate encoders and decoders, FloDR's layout, inverse, and density are properties of a single trained bijection. This ensures that diagnostic visualizations are derived from the exact inverse of the model that generated the layout, not an approximate auxiliary network.
Measurable Claims: The method moves beyond "reading" a map to making measurable claims about it. By introducing the conditional spread and hidden contrast fields, FloDR quantifies what is hidden in the 2D projection.
Falsifiability: The inclusion of a certification process means that diagnostic fields are not just visual overlays but are empirically validated. If a field fails the held-out test, it is refused, preventing the user from drawing unsupported conclusions.
Modest Scope: The authors explicitly state they do not claim superior reconstruction quality (decoders would outperform the inverse) nor do they claim to dominate a single axis (local or global) over all other methods. Instead, FloDR is presented as a method that is highly competitive on both axes simultaneously and provides a rigorous framework for interpreting the limitations of the resulting 2D map.