📊 statistics

Convex relaxation approaches for high-dimensional optimal transport

This paper proposes convex relaxation methods based on marginal and cluster moment statistics to efficiently approximate high-dimensional optimal transport costs with provable convergence rates and error bounds, offering a scalable and interpretable alternative to neural networks for generative modeling.

Original authors: Yuehaw Khoo, Tianyun Tang

Published 2026-07-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuehaw Khoo, Tianyun Tang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Too Many Variables" Puzzle

Imagine you are trying to move a massive pile of sand from one location (let's call it Source) to another (Destination). In the world of mathematics, this is called Optimal Transport (OT). The goal is to find the most efficient way to move every grain of sand so that the total energy spent is minimized.

In a simple world with just a few grains of sand, this is easy. But in modern data science, "grains of sand" can be millions of pixels in an image, thousands of words in a document, or complex genetic data. When the number of variables (dimensions) gets huge, the math breaks down. It's like trying to solve a jigsaw puzzle where the number of pieces grows exponentially with every inch you add to the picture. This is known as the "Curse of Dimensionality."

Standard methods to solve this either take forever to compute or require so much data that you'd need a library the size of a galaxy to get a good answer.

The Solution: The "Local Neighborhood" Strategy

The authors of this paper propose a clever workaround. Instead of trying to solve the entire massive puzzle at once, they break it down into small, manageable neighborhoods.

Think of your data not as one giant, chaotic cloud, but as a city with different districts.

Cluster the City: They group variables that are closely related (like neighbors in the same district) into "clusters."
Look Locally: Instead of tracking how every single person in the city interacts with everyone else, they only look at how people interact within their own district and with their immediate neighbors.
The Relaxation: They use a mathematical trick called Convex Relaxation. Imagine you are trying to find the shortest path through a maze. The exact path is hard to find. Instead, they "relax" the rules slightly to create a simpler, smoother version of the maze that is guaranteed to be at least as short as the real one (a lower bound). This makes the problem solvable by computers.

Two Main Tools: Marginal and Moment Relaxations

The paper introduces two specific ways to do this "local" thinking:

1. Marginal Relaxation (The "Snapshot" Approach)
Imagine you want to understand the traffic flow in a huge country. Instead of tracking every single car, you take snapshots of traffic in specific towns and how those towns connect to their neighbors.

The math ensures these local snapshots are consistent with each other.
It turns the massive problem into a series of smaller, simpler puzzles (Linear Programming problems) that computers can solve instantly.

2. Cluster Moment Relaxation (The "Statistical Summary" Approach)
This is even more powerful for continuous data (like smooth curves rather than discrete dots). Instead of tracking the exact position of every grain of sand, they only track the statistics (moments) of the sand in each neighborhood.

Think of it like describing a crowd not by listing every person's name, but by saying, "In this room, the average height is 5'10", and the average weight is 170 lbs."
By only looking at low-order statistics (averages, variances) within these small clusters, they turn the problem into a Semidefinite Program (SDP). This is a type of math problem that is very stable and efficient to solve, even for huge datasets.

Why This Works: The "Sparse" Advantage

The paper proves that this works incredibly well when the data has a sparse structure.

The Analogy: Imagine a social network where most people only know their immediate family and a few friends, rather than knowing everyone in the world.
The Result: Because the connections are local, the authors show that their method converges (gets the right answer) exponentially fast. This means that even if you only look at a small "radius" of neighbors, you get a result that is almost perfect.
Gaussian Case: For data that follows a bell curve (Gaussian), they proved mathematically that if the connections are sparse, their method is nearly exact and requires far fewer data samples than traditional methods.

Real-World Tests: Does It Actually Work?

The authors didn't just do the math; they tested it on computers with real data:

Toy Gaussian Data: They tested it on simulated data where they knew the exact answer. Their method was much faster and more accurate than standard methods, especially as the data got bigger. While other methods got confused and slow, theirs stayed fast.
Non-Gaussian Data (Beta Distributions): They tested it on weird, non-bell-curve shapes. Even here, their method remained accurate and fast, while standard methods failed as the data size increased.
Ising Models (Physics): They used it to model magnetic spins (like tiny magnets). Their method solved these physics problems in seconds, whereas the exact solution would take hours or days.
Generative Modeling (Creating Images): They used their method to generate new images (like MNIST digits) from random noise.
- They compared their method to Neural Networks (AI models that usually do this).
- The Surprise: Their mathematical approach produced clearer, more accurate images than the neural networks in some cases, and it was much more stable. It offered a simpler, more interpretable alternative to the "black box" of deep learning.

The Takeaway

The paper argues that we don't need to brute-force our way through high-dimensional data with massive neural networks or hope for the best. By realizing that data usually has local structure (things are only strongly connected to their neighbors), we can use convex relaxations to break the problem down.

This approach:

Reduces complexity: Turns impossible problems into solvable ones.
Saves data: Needs fewer samples to get a good answer.
Saves time: Runs much faster than current state-of-the-art methods.
Is interpretable: Unlike neural networks, you can actually see the math behind the solution.

In short, they found a way to solve the "impossible" high-dimensional transport puzzle by only looking at the neighborhood, proving that sometimes, you don't need to see the whole forest to understand the trees.

Technical Summary: Convex Relaxation Approaches for High-Dimensional Optimal Transport

1. Problem Statement

The paper addresses the computational and statistical challenges inherent in solving Optimal Transport (OT) problems in high dimensions. While OT is a powerful framework for comparing probability distributions with geometric fidelity, its application is hindered by the "curse of dimensionality." Specifically, the computational cost of standard OT solvers grows rapidly with sample size, and the sample complexity required to estimate the Wasserstein distance scales exponentially with the dimension $d$ . Existing dimension-reduction strategies, such as neural network parameterizations (which lack strong theoretical guarantees due to non-convex optimization) or sliced Wasserstein distances (which approximate but do not recover the true distance), often fail to provide a scalable, theoretically grounded solution for continuous high-dimensional distributions.

2. Methodology

The authors propose a family of convex relaxation approaches that exploit the locality and sparsity of correlations within high-dimensional distributions. Instead of optimizing over the full joint transport plan $\pi \in \mathcal{P}(X \times Y)$ , the methods approximate the problem using low-order marginals and sparse moment statistics.

2.1 Structural Assumptions

The approach relies on partitioning the $d$ -dimensional coordinates into $K$ clusters and defining a reference graph $G$ that encodes the correlation structure.

Cluster Partition: Coordinates are grouped into clusters $z_k = (x_k, y_k)$ .
Reference Graph: A graph $G$ with vertices representing clusters and edges representing pairwise interactions. The method assumes that dependencies decay with graph distance, allowing for sparse approximations.

2.2 Relaxation Frameworks

The paper introduces two primary levels of relaxation:

Marginal Relaxation:
- The original OT problem is relaxed by enforcing constraints only on the marginals of the transport plan $\pi$ corresponding to single clusters ( $\pi_k$ ) and pairs of connected clusters ( $\pi_{ij}$ ) in the graph $G$ .
- Positivity: The non-negativity of the global measure is relaxed to Positive Semidefinite (PSD) constraints on the block matrix of these marginals.
- Formulations: The authors define three variants:
  - $(OT_{mar})$ : Includes PSD constraints and non-negativity (Doubly Non-Negative programming).
  - $(OT^1_{mar})$ : Drops the global PSD constraint, resulting in a Linear Programming (LP) problem.
  - $(OT^2_{mar})$ : Drops the non-negativity constraint, resulting in a Semidefinite Program (SDP) that can be further reduced via chordal conversion.
Cluster Moment Relaxation:
- To handle continuous distributions without spatial discretization, the marginal relaxations are further approximated using Sum-of-Squares (SOS) moment relaxations.
- Instead of optimizing over measures, the variables are finite-dimensional moment matrices ( $M_k, M_{ij}$ ) derived from a polynomial basis of degree $n$ .
- Key Innovation: Unlike standard SOS hierarchies (e.g., Mula and Nouy [44]) which require moments of all monomials up to degree $n$ (leading to $O(d^{2n})$ complexity), this method restricts the basis to local clusters and edges of the reference graph.
- Complexity: The resulting SDP involves only sparse collections of moments. If the cluster size and relaxation degree are fixed, the problem size scales linearly with $d$ (specifically $O(d)$ ) when the graph $G$ has bounded tree-width, solvable via chordal decomposition.

2.3 Transport Map Extraction

The dual variables of the relaxed SDP provide approximate Kantorovich potentials. By restricting these potentials to the span of the chosen polynomial basis, the authors construct an explicit transport map $\hat{T}(x) = x - \frac{1}{2}\nabla \hat{f}(x)$ , offering an interpretable alternative to neural network-based generative models.

3. Key Contributions and Theoretical Results

3.1 Gaussian Models with Sparse Precision

For Gaussian measures $\mu = \mathcal{N}(m_1, \Sigma_1)$ and $\nu = \mathcal{N}(m_2, \Sigma_2)$ where the precision matrices are sparse with respect to a graph $G$ :

Exponential Convergence: The authors prove that the cluster moment relaxation (specifically degree-1) achieves an exponential convergence rate in the approximation error as the connectivity radius $h$ of the reference graph $G_h$ increases. The error bound is of the form $C d \rho^{-h}$ .
Improved Sample Complexity: The statistical error bound for the sparse relaxation is $O(\sqrt{d/N} + |E(G)|/N)$ . For sparse graphs where $|E(G)| = O(d)$ , this improves upon the dense relaxation bound of $O(\sqrt{d/N} + d^2/N)$ , demonstrating that exploiting sparsity significantly reduces the number of samples required.

3.2 Local Perturbations of Mean-Field Measures

For distributions that are local perturbations of mean-field product measures:

Approximation Bounds: The marginal relaxation is shown to capture the leading-order geometry of the Wasserstein distance, which corresponds to the weighted negative homogeneous Sobolev norm.
Statistical Error: Under fixed relaxation degree and bounded cluster size, the statistical error scales as $O(d N^{-1/2})$ , implying a sample complexity of $O(d^2 \epsilon^{-2})$ . This is a significant improvement over the exponential dependence on $d$ found in standard OT estimation.

3.3 General Statistical Bounds

For general distributions satisfying uniform Gaussian tail conditions, the paper establishes that the statistical error of the cluster moment SDP is governed by the number of local blocks and graph edges ( $K + |E(G)|$ ) rather than the ambient dimension $d$ . In the sparse fixed-degree regime, the error scales linearly with $d$ .

4. Numerical Results

Extensive experiments validate the theoretical claims across various settings:

Gaussian Distributions: The cluster moment relaxation exhibits exponential convergence with respect to the connectivity radius $h$ . It outperforms vanilla OT solvers (Sinkhorn) and ICNN baselines in both accuracy and computational efficiency, maintaining low relative error and running time even as dimension $d$ increases to 1000.
Non-Gaussian Product Measures (Beta): The method maintains high accuracy (relative error $\approx 10^{-3}$ ) for non-Gaussian product measures, whereas vanilla OT solvers degrade rapidly with dimension.
Ising Models: For discrete Ising models, the marginal relaxation provides tight lower bounds on the OT cost with running times orders of magnitude faster than exact LP solutions.
Generative Modeling (Ginzburg-Landau & MNIST): The extracted transport maps successfully generate samples that match target marginals. In Ginzburg-Landau models, the method outperforms normalizing flows, ICNNs, and Sinkhorn-based maps in sliced Wasserstein distance. On MNIST (in a 6D PCA space), the method produces recognizable digit samples.

5. Significance and Claims

The paper claims that convex relaxations based on marginal and cluster moments offer a promising path for dimension reduction in high-dimensional optimal transport.

Scalability: By leveraging local structures and sparsity, the method avoids the exponential complexity of standard OT and full moment hierarchies, achieving linear scaling in dimension for both error and running time under specific structural assumptions.
Theoretical Guarantees: Unlike neural network approaches, the proposed method provides rigorous approximation and statistical error bounds, particularly for sparse Gaussian and mean-field perturbation models.
Interpretability: The approach yields explicit transport maps derived from polynomial potentials, serving as a simpler, interpretable alternative to deep generative models.
Robustness: The empirical results suggest the method is robust even in non-Gaussian settings where theoretical guarantees are less explicit, maintaining performance where traditional discretization-based methods fail.

The authors conclude that these relaxations effectively mitigate the curse of dimensionality, making high-dimensional OT computationally and statistically tractable for a broad class of structured distributions.