📊 statistics

Exact Stiefel Optimization for Probabilistic PLS: Closed-Form Updates, Error Bounds, and Calibrated Uncertainty

This paper introduces an end-to-end framework for Probabilistic Partial Least Squares that overcomes existing optimization bottlenecks by combining noise-subspace estimation with exact Stiefel-manifold optimization, achieving minimax-optimal convergence, closed-form uncertainty calibration, and superior predictive performance on multi-omics benchmarks.

Original authors: Haoran Hu, Xingce Wang

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoran Hu, Xingce Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding the Signal in the Noise

Imagine you are trying to understand a conversation between two people, Alice (View X) and Bob (View Y). They are speaking in different languages, but they are discussing the same underlying topics (the "latent factors"). However, their conversations are filled with static, background noise, and they sometimes talk about things unique to themselves that the other person doesn't know about.

Probabilistic Partial Least Squares (PPLS) is a mathematical tool designed to figure out:

What are the shared topics they are discussing?
How much of what they say is just random noise?
If Alice says something new, what is the most likely thing Bob will say, and how confident can we be in that prediction?

This paper introduces a new, faster, and more reliable way to solve this puzzle, especially when the "static" (noise) is very loud.

The Problem with the Old Way

Previously, scientists used a method called EM/ECM to solve this. Think of this like trying to tune a radio while the volume knob is broken.

The Tangle: The old method tried to figure out the "topics" (signal) and the "static" (noise) at the same time. Because they were tangled together, if the static was loud, the method got confused. It would guess the topics wrong, which made it guess the static wrong, which made it guess the topics even worse. It was a messy loop.
The Constraint: The math requires the "topics" to be perfectly independent of each other (like distinct, non-overlapping channels). The old method tried to force this rule using a "penalty" (like a soft brake), which often resulted in the channels being slightly messy or overlapping, leading to errors.

The New Solution: "Fix the Static First"

The authors propose a new pipeline that breaks the problem into three clear steps, like a professional audio engineer fixing a recording:

1. The Noise Subspace Estimator (Cleaning the Static First)

Instead of guessing the noise while trying to hear the music, this method listens to the "silence" first.

The Analogy: Imagine a crowded room. If you want to know how loud the background chatter is, you don't listen to the people talking; you listen to the empty corners where no one is speaking.
The Innovation: The authors realized that the "noise" lives in the empty corners of the data (the low-eigenvalue subspace). By measuring only that empty space, they get a perfect estimate of the noise level.
Why it matters: The old method tried to measure noise by averaging everything (including the music), which made the noise estimate too high and biased. The new method is mathematically proven to be accurate regardless of how loud the music (signal) is.

2. Exact Stiefel Optimization (The Perfect Dance Floor)

Once the noise level is fixed and known, the method focuses purely on finding the shared topics.

The Analogy: The math requires the topics to be "orthogonal" (at right angles to each other, like the X, Y, and Z axes). The old method tried to keep them at right angles by pushing them back if they drifted (a penalty). The new method treats the problem as a dance on a specific, curved floor (a Stiefel manifold).
The Benefit: On this "dance floor," every step the algorithm takes is guaranteed to keep the topics perfectly at right angles. There is no drifting, no messy corrections, and no "soft brakes." It's a precise, geometric walk.

3. Calibrated Uncertainty (The Weather Forecast)

Most models give you a prediction (e.g., "It will rain"). Good models also tell you how sure they are (e.g., "It will rain, 90% chance").

The Innovation: Because this method is so precise about the noise and the geometry, it can naturally calculate how confident it is in its predictions.
The Result: When the authors tested this, the confidence intervals were "calibrated." If the model said "95% confidence," it was right 95% of the time. Other methods (like deep learning) often get this wrong and need extra, messy "post-processing" to fix their confidence levels.

The "Gaussianization" Trick (Optional)

Sometimes the data isn't perfectly "bell-curve" shaped (Gaussian). The authors added an optional step called Gaussianization.

The Analogy: If the data is like a lumpy potato, this step turns it into a smooth egg shape without changing the core relationships. This allows the math to work perfectly even if the raw data is weird or non-standard.

Real-World Tests: What Happened?

The authors tested this on two types of data:

Synthetic Data: Made-up data where they knew the "truth."
- Result: Their method recovered the true signals much better than the old methods, especially when the noise was very high. It was also much faster.
Real Data (TCGA-BRCA & CITE-seq):
- TCGA-BRCA (Breast Cancer): They tried to predict one type of biological data from another. Their method was as accurate as the best standard methods but provided reliable confidence intervals without needing extra fixes.
- CITE-seq (Cell Biology): This data is very complex. While deep learning models (neural networks) were slightly better at raw prediction accuracy, they were terrible at knowing how sure they were. The new method was almost as accurate but gave honest, reliable confidence scores and explained which factors were driving the results (interpretability).

Summary of the "Win"

Old Way: Tangled signal and noise, messy constraints, often overconfident or underconfident predictions.
New Way:
1. Measures noise by looking at the empty space (accurate).
2. Optimizes on a perfect geometric floor (stable and fast).
3. Gives honest confidence scores automatically (reliable).

The paper concludes that this approach is a "plug-and-play" upgrade for anyone doing two-view learning who needs not just a prediction, but a trustworthy measure of uncertainty.

Technical Summary: Exact Stiefel Optimization for Probabilistic PLS

Problem Statement
Probabilistic Partial Least Squares (PPLS) is a likelihood-based framework for two-view learning that provides interpretable latent factors and calibrated uncertainty. However, existing fitting pipelines, typically relying on Expectation-Maximization (EM) or ECM updates, face two critical bottlenecks:

Noise–Signal Coupling: Joint updates of latent signal parameters and noise variances create mutual dependencies that flatten the objective function in certain directions under high noise, leading to initialization sensitivity and parameter recovery errors.
Orthogonality Constraints: Enforcing the orthonormality of loading matrices ( $W^\top W = C^\top C = I_r$ ) within EM/ECM substeps is nontrivial. Existing methods often handle this indirectly via interior-point penalties, which do not guarantee exact feasibility at every iteration.

Furthermore, previous approaches to noise estimation (e.g., Hu et al., 2025) often average over the full spectrum of eigenvalues, mixing signal and noise directions, which introduces deterministic bias and inconsistency.

Methodology
The authors propose an end-to-end framework that decouples noise estimation from constrained signal optimization, operating within the identifiable PPLS parameterization of Bouhaddani et al. (2018). The pipeline consists of four stages:

Noise-Subspace Pre-estimation: Instead of full-spectrum averaging, the method estimates observation noise variances ( $\sigma^2_e, \sigma^2_f$ ) by averaging the smallest $p-r$ and $q-r$ eigenvalues of the marginal sample covariances. This targets the noise subspace directly, avoiding contamination from signal eigenvalues.
Fixed-Noise Scalar Likelihood: With noise variances fixed, the observed-data negative log-likelihood is reduced to a scalar objective function. This expansion avoids explicit matrix determinants and inverses, expressing the likelihood in terms of projected scalar statistics ( $Q_x, Q_y, Q_{xy}$ ) and diagonal parameters.
Exact Stiefel-Manifold Optimization: The constrained optimization is performed on the product manifold $\text{St}(p, r) \times \text{St}(q, r) \times \mathbb{R}^{2r+1}_{++}$ $St (p, r) \times St (q, r) \times R_{++}^{2 r + 1}$ . The paper introduces two strategies:
- SLM-Manifold: A full Riemannian optimization approach using conjugate gradients and QR retractions to maintain exact orthogonality at every iterate.
- BCD-SLM: A block-coordinate descent scheme exploiting the scalar separability of the PPLS objective. It updates loading matrices via Riemannian steps and updates scalar parameters ( $\theta^2_{t,i}, b_i, \sigma^2_h$ ) via closed-form conditional maximizers (solving cubic equations for $b_i$ ).
Gaussianization and Calibration: An optional coordinate-wise Gaussianization layer (e.g., Rank-based Inverse Normal Transform) extends applicability to sub-Gaussian observations. Prediction uncertainty is calibrated via a dispersion factor $\hat{\kappa}$ derived from validation residuals, ensuring near-nominal coverage without post-hoc recalibration.

Key Contributions

Rate-Optimal Noise Estimation: The authors prove that the proposed noise-subspace estimator achieves a finite-sample error rate independent of signal strength (Theorem 1) and matches a minimax lower bound (Theorem 2). Conversely, they demonstrate that the full-spectrum estimator used in prior work is inconsistent, suffering from a deterministic bias that does not vanish with sample size (Proposition 1).
Exact-Manifold Optimization with Inference Guarantees: By optimizing on the exact Stiefel manifold, the method ensures strict orthogonality feasibility. The resulting estimator is shown to be consistent and asymptotically normal. Crucially, the Fisher information matrix exhibits a block-diagonal structure, allowing for the derivation of closed-form standard errors for the parameters without Monte Carlo bootstrapping (Proposition 6).
Extended Modeling Scope: The framework accommodates non-Gaussian data via optional Gaussianization, interpreting the likelihood as a Gaussian quasi-likelihood. Theoretical analysis shows this reduces excess-kurtosis bias to $O(1/N)$ (Corollary 2).
Rank Selection: A two-stage procedure is proposed for data-driven rank selection using BIC, with theoretical guarantees on consistency and weak-factor detectability thresholds (Propositions 7 and 8).

Experimental Results

Synthetic Performance: In high-noise regimes, the proposed fixed-noise solvers (SLM-Manifold and BCD-SLM) significantly outperform EM/ECM in recovering coupling parameters ( $B, \Sigma_t$ ) and loading matrices, particularly when $p=q=500$ . The methods achieve near-nominal predictive interval coverage (e.g., 95% coverage at 95% nominal level) without post-hoc adjustment.
Real-World Benchmarks:
- TCGA-BRCA: The method achieves point-prediction accuracy comparable to Ridge regression (MSE $\approx$ 0.45) with only $r=3$ components, while providing native calibrated uncertainty. In contrast, PO2PLS (which adds view-specific orthogonal components) achieves similar MSE but lacks native predictive uncertainty and does not outperform the fixed-noise approach.
- PBMC CITE-seq: While nonlinear deep baselines (DCCA, KCCA) achieve slightly lower point MSE, they require extensive post-hoc recalibration to achieve usable uncertainty. The fixed-noise PPLS pipeline provides competitive accuracy (MSE $\approx$ 0.259) with model-native calibration and interpretable factors, avoiding the need for external recalibration stages.
Computational Efficiency: The scalar-form optimization offers an $O(p/r)$ speedup over matrix-form evaluations. BCD-SLM converges faster than full Riemannian optimization, with runtimes in the seconds-to-tens-of-seconds range for $p=q=500$ .

Significance and Claims
The paper claims that direct optimization of the PPLS likelihood on the product Stiefel manifold, combined with spectrally pre-estimated noise, yields a practical linear probabilistic pipeline with model-native uncertainty calibration.

The authors position their work as a shift from "best single calibration number" (often achieved by deep models only after fragile post-hoc repair) toward deployment simplicity, interpretability, and reduced calibration fragility. They argue that while PO2PLS is superior for decomposing view-specific structures, the fixed-noise PPLS pipeline is the appropriate tool when the goal is calibrated cross-view prediction. The method is presented as a robust alternative to EM/ECM in high-noise settings and a more reliable uncertainty quantifier than deep learning baselines that lack native probabilistic structure.

The authors remain modest regarding open problems, noting that a global landscape analysis for the general PPLS case (beyond the PCCA submanifold) remains open, as does the development of a unified noise estimator that is both consistent and rank-agnostic.