📊 statistics

Partial Identification under High-Dimensional Potential Outcomes and Confounders via Optimal Transport

This paper proposes a novel estimator for partial identification in high-dimensional causal settings that overcomes the curse of dimensionality by decomposing the optimal transport problem into a low-dimensional signal subspace and a high-dimensional residual subspace, where the latter is efficiently recovered using the Sliced Wasserstein distance to yield tighter, more informative causal bounds.

Original authors: Yunfeng Wang, Zhiheng Zhang, Zijun Gao

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yunfeng Wang, Zhiheng Zhang, Zijun Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Missing Puzzle Piece" Problem

Imagine you are a detective trying to figure out the true effect of a new medicine. You have data on patients who took the medicine and those who didn't. However, there's a catch: for every patient, you can only see one outcome (what happened after they took the medicine, or what happened if they didn't), but never both at the same time.

In statistics, this is called Partial Identification. You can't pinpoint the exact answer (the "point identification"), but you can draw a box around the possible answers. The goal is to make that box as small and tight as possible so your decision is more precise.

To make this box tighter, you look at other factors (like age, weight, or blood pressure) called confounders. If you can group patients who are very similar, you can get a better estimate.

The Problem: The "High-Dimensional" Trap

The paper argues that modern data is often high-dimensional. Imagine instead of just looking at age and weight, you are looking at 30 different health metrics, or even thousands of genetic markers.

When you try to compare patients across 30 or 100 dimensions, standard mathematical tools break down. It's like trying to find the shortest path between two cities on a map that has 1,000 dimensions instead of 2. The math gets so messy and computationally heavy that the results become useless or wildly inaccurate. This is known as the "curse of dimensionality."

Existing methods try to fix this by ignoring most of the data. They say, "Let's just look at the top 5 most important factors and throw the rest away."

The Analogy: Imagine you are trying to guess the total weight of a suitcase. You weigh the heavy books (the "signal") but decide to ignore the clothes, socks, and toiletries (the "residual") because there are too many of them. You get a number, but it's definitely too low because you threw away a lot of weight.

The Solution: The "Signal and Slicing" Method (CSS)

The authors propose a new method called Conditioned Subspace–Slicing (CSS). Instead of throwing away the "leftover" data, they find a clever way to weigh it without doing the impossible math.

Here is how it works, broken down into two steps:

1. The Signal Subspace (The Heavy Books)

First, the method identifies the few dimensions where the biggest differences between the two groups (medicine vs. no medicine) actually happen.

Analogy: You find the heavy books in the suitcase and weigh them precisely. This is the "Signal."

2. The Sliced Residual (The Clothes)

This is the paper's innovation. Instead of ignoring the clothes (the "Residual"), they use a technique called Sliced Wasserstein Distance.

The Analogy: Imagine you can't weigh the whole pile of clothes at once. So, you take a knife and slice the suitcase into thin, one-dimensional strips (like slicing a loaf of bread). You weigh each slice individually.
Why this works: Weighing a single slice is easy and fast, even if the suitcase is huge. By averaging the weights of all these random slices, you get a very good estimate of the total weight of the clothes.
The Result: You add the precise weight of the books (Signal) to the estimated weight of the clothes (Residual).

Why This Is Better

The paper proves mathematically that this new method is:

Valid: It never overestimates the effect. It always provides a "safe lower bound" (a conservative estimate that is guaranteed to be true).
Tighter: Because it recovers the weight of the "clothes" (the residual data) that other methods throw away, the final box of possible answers is much smaller and more informative.
Efficient: It doesn't require supercomputers. The "slicing" trick makes the math fast enough to run on high-dimensional data.

The "Spiked" Assumption

The method works best under a specific condition the authors call the Extended Spiked Transport Model.

Analogy: Imagine the suitcase is mostly empty space, but it has a few very dense, heavy objects (the signal) and a lot of fluffy, evenly distributed cotton (the residual).
If the "fluffy cotton" is spread out evenly (isotropic), the "slicing" method works perfectly to estimate its weight. The paper shows that in many real-world scenarios, the "noise" or leftover data behaves like this fluffy cotton, making the method very effective.

Summary of Results

The authors tested this on:

Fake Data: They created a scenario where they knew the exact answer. The new method (CSS) got much closer to the true answer than the old methods (which just looked at the top 5 factors).
Real Data: They used a medical dataset about heart catheterization. Even without knowing the "true" answer, the new method produced a tighter, more useful range of possibilities than the old methods.

In short: The paper gives us a new tool to solve complex causal questions in high-dimensional data. Instead of ignoring the messy, complex parts of the data, it uses a "slicing" trick to estimate them efficiently, resulting in much sharper and more reliable answers.

Technical Summary: Partial Identification under High-Dimensional Potential Outcomes and Confounders via Optimal Transport

1. Problem Statement

The paper addresses the challenge of partial identification (PI) in causal inference when both potential outcomes and confounders are high-dimensional. In many causal settings, point identification is impossible because the joint distribution of potential outcomes is not determined by observed data. While pre-treatment covariates can tighten the set of feasible causal effects (the PI set), existing methods based on Optimal Transport (OT) struggle in high-dimensional regimes.

Specifically, classical OT-based bounds suffer from the curse of dimensionality, leading to statistically unstable estimates and unfavorable convergence rates. Existing covariate-assisted PI frameworks often face a trade-off:

Coarse stratification leads to overly conservative bounds.
Direct plug-in estimation of conditional Wasserstein distances is computationally prohibitive and statistically fragile.
Projection-only methods (e.g., Wasserstein Projection Pursuit) discard residual information, yielding systematically loose lower bounds when non-negligible transport discrepancy exists in the orthogonal complement of the projected subspace.

The core problem is constructing a dimension-adaptive, statistically stable transport quantity that serves as a valid lower bound for the quadratic Wasserstein distance in high dimensions without sacrificing theoretical validity.

2. Methodology: Conditioned Subspace–Slicing (CSS)

The authors propose a novel estimator called the Conditioned Subspace–Slicing (CSS) certificate. This approach decomposes the transport problem into two components: a low-dimensional signal subspace and a high-dimensional residual subspace.

2.1 The CSS Objective

For a candidate $r$ -dimensional subspace $V$ and its orthogonal complement $V^\perp$ , the CSS objective $L(V)$ is defined as:
$L(V) = W_2^2(\mu_V, \nu_V) + k \cdot SW_2^2(\mu_{V^\perp}, \nu_{V^\perp})$
where:

$W_2^2(\mu_V, \nu_V)$ is the squared quadratic Wasserstein distance on the projected signal subspace $V$ .
$SW_2^2(\mu_{V^\perp}, \nu_{V^\perp})$ is the Sliced Wasserstein distance on the residual subspace $V^\perp$ .
$k = d - r$ is a scaling factor (dimension of the residual space).

2.2 Theoretical Justification

Validity: Theorem 3.1 proves that $L(V)$ is a valid lower bound for the true squared Wasserstein distance $W_2^2(\mu, \nu)$ . The scaling by $k$ is crucial; without it, the sliced term decays as $d^{-1}$ and becomes vacuous. With scaling, it provides a certified recovery of residual transport energy.
Tightness: Under an Extended Spiked Transport Model (ESTM), where transport energy concentrates on a low-dimensional signal subspace and the residual is approximately isotropic, the CSS bound is near-tight. The gap between the true distance and the bound is governed by a single residual anisotropy deficit.
Subspace Stability: Theorem 3.6 demonstrates that under the ESTM, the subspace that maximizes the CSS objective remains close to the true signal subspace (identified by the signal eigenvalues), provided there is a sufficient eigengap between the signal and residual components.

2.3 Estimation Algorithm

The proposed plug-in estimator (Algorithm 1) follows a two-stage strategy:

Signal Recovery: Use Wasserstein Projection Pursuit (WPP) to find the optimal $r$ -dimensional subspace $\hat{U}$ that maximizes the projected Wasserstein distance.
Residual Recovery: Compute the Sliced Wasserstein distance on the orthogonal complement $\hat{U}^\perp$ using Monte Carlo sampling of random one-dimensional projections.
Aggregation: Combine the signal term and the scaled residual term to form the final lower bound.

3. Key Contributions

The paper makes three primary contributions:

A Certified Lower Bound: It introduces the CSS lower bound for quadratic Wasserstein distances, which serves as a dimension-adaptive partial identification region. It yields certified conditional and unconditional transport lower bounds in high dimensions.
Structural Theory: It formalizes the Extended Spiked Transport Model (ESTM), under which the CSS bound is near-tight and the optimal subspace is identified by WPP. It establishes that the approximation gap is controlled by the anisotropy of the residual transport.
Finite-Sample Guarantees: It provides non-asymptotic error bounds for the CSS estimator. These bounds scale with the intrinsic dimension (signal rank) rather than the ambient dimension, ensuring statistical tractability. The error analysis separates the intrinsic-dimensional rate of the signal term from the parametric rate of the sliced residual term.

4. Experimental Results

The authors evaluate the CSS estimator against WPP (projection-only) and Full Sliced Wasserstein (averaging over the full ambient space) baselines.

Synthetic Data:
- In a Gaussian pushforward setting ( $d=30$ , signal rank $r=5$ ), the CSS estimator consistently stays closer to the ground truth $W_2^2$ than WPP.
- WPP underestimates the distance by discarding residual energy, while the Full Sliced baseline underestimates it due to averaging over random directions in the full space.
- CSS successfully recovers the "lost" transport energy via the scaled sliced term, even as the residual anisotropy parameter increases.
- Finite-sample analysis shows that the subspace recovery error decreases consistently with sample size $n$ , demonstrating stability.
Real Data (RHC Dataset):
- Applied to a right-heart catheterization dataset with a 27-dimensional outcome vector.
- CSS produces higher (more informative) held-out lower certificates than WPP across various subspace dimensions.
- The gap between CSS and WPP is most pronounced at smaller dimensions where more outcome variation remains in the orthogonal complement.
- Both methods outperform the full scaled sliced-Wasserstein baseline, highlighting the importance of learning a task-adaptive signal subspace.

5. Significance and Claims

The paper claims to provide a modular, dimension-adaptive geometric primitive that can be integrated into existing covariate-assisted partial identification frameworks.

Complementarity: The work is complementary to prior research on OT-based PI (e.g., Ji et al., 2024; Lin et al., 2025). While previous works focus on inferential robustness, relaxation tightness, and topological consistency, this paper addresses the statistical bottleneck of high-dimensional transport estimation.
"Free Lunch": The authors characterize the method as a principled "free lunch": it replaces a high-dimensional transport problem with a combination of low-dimensional OT and inexpensive sliced computations. Under mild conditions (residual isotropy), this recovers the full transport energy without loosening the bound.
Robustness: The method offers a robust alternative to projection-only strategies, which systematically discard residual information, and direct high-dimensional estimation, which is statistically unstable.

The paper concludes that the CSS estimator effectively closes the population gap at the signal scale with minimal finite-sample overhead, offering a robust primitive for high-dimensional causal inference where point identification is unattainable.