Technical Summary: Missing-Data-Induced Phase Transitions in Spectral PLS for Multimodal Learning

Problem Statement

Multimodal learning often involves extracting shared low-dimensional structures from paired data views $(X, Y)$. While Partial Least Squares (PLS), specifically its spectral formulation via Singular Value Decomposition (PLS-SVD), is a standard tool for this task, real-world multimodal datasets frequently suffer from missing entries in both views due to acquisition failures or heterogeneous pipelines.

The central problem addressed is understanding the fundamental limits of PLS-SVD when both views are subject to Missing-Completely-At-Random (MCAR) masking. Specifically, the authors investigate:

1. Under what conditions does PLS-SVD fail to recover the latent shared structure when data is missing?
2. How does the missingness mechanism quantitatively alter the recovery threshold and the asymptotic overlap between estimated and true singular vectors?
3. Does a sharp phase transition exist where the estimator shifts from being uninformative to informative as the signal strength increases?

Methodology

Theoretical Framework

The authors analyze PLS-SVD within a proportional high-dimensional spiked model.

• Setup: Let $N$ be the number of samples and $D_x, D_y$ be the feature dimensions of the two views. The aspect ratios are $\alpha_x = N/D_x$ and $\alpha_y = N/D_y$.
• Signal Model: The complete (unmasked) data follows a rank-1 spiked model where $Y_\star = \theta (X_\star u_0)v_0^\top + Z$, with $X_\star$ whitened, $u_0, v_0$ being unit signal vectors, and $Z$ being i.i.d. Gaussian noise.
• Missingness: Independent MCAR masks $S_x$ and $S_y$ are applied, with retention probabilities $\rho_x$ and $\rho_y$. The joint retention probability is $\rho = \rho_x \rho_y$. Missing entries are treated as zero.
• Estimator: The algorithm computes the leading singular vectors of the rescaled empirical cross-covariance matrix $C = \frac{1}{N\sqrt{\rho}} X^\top Y$.

Analytical Approach

The paper employs Replica Symmetry (RS) analysis, a technique from statistical physics used to study high-dimensional random matrix problems.

1. Reduction to Spiked Model: The authors demonstrate that under dual MCAR masking, the observed cross-covariance matrix behaves asymptotically like a spiked rectangular random matrix. Crucially, the missingness acts as a multiplicative attenuation of the signal strength, defining an effective spike $\theta_{\text{eff}} = \sqrt{\rho} \theta$.
2. Free Energy Optimization: By introducing source fields and taking the zero-temperature limit ($\beta \to \infty$), the authors derive a replica-symmetric free energy density. This is optimized to find the stationary points corresponding to the asymptotic overlaps.
3. Phase Transition Derivation: The analysis yields a critical threshold condition. Below this threshold, the leading singular vectors are asymptotically orthogonal to the true signal (zero overlap). Above it, non-trivial alignment is achieved.

Key Contributions

1. Sharp Phase Transition Prediction: The paper derives a closed-form critical threshold $\theta_{\text{crit}}$ for PLS-SVD under dual missingness:
   $$\theta_{\text{crit}} = \frac{1}{(\alpha_x \alpha_y)^{1/4} \sqrt{\rho}}$$
   This establishes that missing data effectively increases the required signal-to-noise ratio by a factor of $1/\sqrt{\rho}$.

2. Asymptotic Overlap Formulas: For signal strengths $\theta > \theta_{\text{crit}}$, the authors provide explicit formulas for the asymptotic squared overlaps $r_x^2$ and $r_y^2$ between the estimated and true singular vectors. These formulas depend on the aspect ratios, the joint retention probability, and the signal strength.

3. Finite-Rank Extension (Conjecture): The authors conjecture that for rank-$k$ latent structures with separated spikes, the single-spike threshold applies component-wise. That is, the $i$-th component is recoverable if and only if its specific strength $\theta_i$ exceeds the adjusted threshold $\theta_{\text{crit}}$.

4. Empirical Validation:

   • Synthetic Experiments: Extensive Monte Carlo simulations confirm the predicted phase boundary and overlap curves across varying aspect ratios, signal strengths, and missingness rates.
   • Semi-Synthetic Experiments: Using real biological datasets (TCGA BRCA and PBMC Multiome), the authors extract empirical signal directions and validate that the theoretical phase transition holds even when the signal geometry is non-random and biological.
   • Practical Diagnostics: The paper proposes a "split-half stability" metric (correlation between singular vectors from two random data halves) as a ground-truth-free diagnostic to detect whether the system is in the recoverable regime.

Results

• Attenuation Effect: The primary finding is that dual missingness reduces the effective signal-to-noise ratio by $\sqrt{\rho}$. For example, a 30% dropout rate in both views ($\rho \approx 0.49$) requires a $1/\sqrt{0.49} \approx 1.43\times$ increase in signal strength to maintain the same recovery performance as the fully observed case.
• Phase Diagram: The transition from uninformative to informative recovery is sharp. The theoretical phase boundary matches empirical heatmaps with high correlation ($r > 0.99$).
• Robustness:
  • The phase transition location is robust to moderate deviations from MCAR (e.g., Missing-At-Random), though recovery quality degrades if missingness is signal-dependent.
  • The theory holds for various noise distributions, though infinite-kurtosis noise (e.g., Student-t with $\nu=3$) degrades supercritical overlaps.
  • Simple "missing-as-zero" PLS-SVD with appropriate normalization performs competitively against more complex imputation baselines (EM-PLS, Iterative SVD) under MCAR, offering superior computational efficiency.
• Preprocessing: For count-valued data (e.g., RNA-seq), variance-stabilizing transformations (rank normalization, sqrt-CPM) are necessary before whitening to align with the theoretical predictions; raw or log-CPM transformations lead to deviations.

Significance and Claims

The paper claims to provide the first rigorous characterization of the fundamental limits of spectral PLS in the presence of missing data in both views. By framing the problem within the context of spiked random matrix theory and the BBP (Baik-Ben Arous-Péché) transition, the authors offer:

1. A Predictive Tool: A precise formula to determine whether a given multimodal dataset with missing entries contains enough signal for PLS-SVD to succeed.
2. A Mechanistic Understanding: The identification of missingness as a simple multiplicative attenuation of the signal, rather than a complex structural corruption, simplifies the analysis of missing-data problems in spectral methods.
3. Practical Guidance: The work suggests that in high-dimensional settings, simple normalization of missing-as-zero data is often sufficient, provided the signal strength is adjusted for the retention probability. It also highlights that split-half stability can serve as a practical proxy for the theoretical phase transition when ground truth is unavailable.

The authors remain modest regarding limitations, noting that the analysis assumes MCAR, i.i.d. Gaussian noise, and a whitened design. They acknowledge that the finite-rank extension is a conjecture supported by experiments but not rigorously proven, and that the argument does not directly transfer to Canonical Correlation Analysis (CCA) where masking perturbs within-view covariance geometry.