Inference on covariance structure in high-dimensional multi-view data

1. Problem Statement

The paper addresses the challenge of estimating covariance structures in high-dimensional multi-view data (e.g., multi-omics datasets where different data types like gene expression, methylation, and protein levels are collected on the same samples).

Key Challenges:

• Heterogeneity: Different views often have vastly different dimensions ($p_m$), distributions, and signal-to-noise ratios.
• Limitations of Existing Methods:
  • Bayesian MCMC/VI: Popular factor-analytic approaches (e.g., MOFA) rely on Markov Chain Monte Carlo (MCMC) or Variational Inference (VI). MCMC is computationally expensive and slow to converge in high dimensions. VI often underestimates uncertainty and lacks theoretical guarantees.
  • Concatenation: Simply concatenating views ignores the multi-view structure, leading to sub-optimal results, especially when views have low signal-to-noise ratios (they get "drowned out" by high-dimensional, high-signal views).
  • JIVE: Matrix factorization methods like JIVE provide point estimates but lack uncertainty quantification.
• Goal: Develop a scalable method that provides accurate point estimates, valid uncertainty quantification (credible intervals), and rigorous theoretical guarantees for both intra-view and inter-view covariances without expensive sampling.

2. Methodology: FAMA

The authors propose FAMA (Factor Analysis for Multi-view data via spectral Alignment). The method decouples the estimation of latent factors from the inference of loadings and variances.

Step 1: Spectral Estimation and Alignment of Latent Factors

Instead of inferring which factors are active in which view (a difficult combinatorial problem), FAMA estimates the global latent factors active in at least one view.

1. Per-View SVD: For each view $m$, compute the Singular Value Decomposition (SVD) of the data matrix $Y_m$. Let $P_m = U_m U_m^\top$ be the projection matrix onto the space spanned by the top $k_m$ singular vectors.
2. Averaging Projections: Compute the empirical average of these projection matrices: $\tilde{P} = \frac{1}{M} \sum_{m=1}^M P_m$.
3. Global Factor Estimation: Perform an SVD on $\tilde{P}$. The top $k_0$ left singular vectors ($U$) of $\tilde{P}$ provide an estimate of the global latent factors $\hat{F} = \sqrt{n}U$.
   • Theoretical Insight: This step aligns the subspaces from different views. Theorem 1 proves that the Procrustes error between $\hat{F}$ and the true factors $F_0$ vanishes as $n$ and $p_m$ diverge.

Step 2: Surrogate Bayesian Regression for Loadings

Once $\hat{F}$ is estimated, it is treated as an observed covariate. The model becomes a standard regression problem for each variable in each view:
$$\hat{Y}_m = \hat{F} \tilde{\Lambda}_m^\top + \tilde{E}_m$$

• Priors: Conjugate Normal-Inverse Gamma priors are placed on the factor loadings ($\tilde{\Lambda}$) and residual variances ($\tilde{\sigma}^2$).
• Posterior: Because the priors are conjugate and the "covariates" ($\hat{F}$) are fixed, the posterior distribution for each variable's parameters is a product of Normal-Inverse Gamma distributions. This allows for analytical, parallel computation of the posterior, bypassing MCMC entirely.

Step 3: Variance Inflation for Valid Coverage

The raw posterior derived from treating $\hat{F}$ as fixed tends to under-cover (intervals are too narrow) because it ignores the uncertainty in estimating $\hat{F}$.

• Correction: The authors introduce a variance inflation factor $\rho_m > 1$.
• Calibration: $\rho_m$ is chosen based on the magnitude of the estimated loadings and residual variances (Equation 14) to ensure asymptotically valid frequentist coverage for the credible intervals.

Step 4: Hyperparameter Selection

• Latent Dimensions ($k_m, k_0$): Estimated using a Joint Information Criterion (JIC) and a gap statistic on the singular values of $\tilde{P}$.
• Prior Variances: Data-adaptive strategies are used to set prior hyperparameters based on the observed signal strength.

3. Key Contributions

1. Spectral Alignment: A novel procedure to estimate and align latent factors across heterogeneous views using averaged projection matrices, avoiding the need to identify view-specific factor activity a priori.
2. Scalable Bayesian Computation: A method that yields full posterior distributions for covariance structures via analytical formulas (Normal-Inverse Gamma), eliminating the need for slow MCMC sampling.
3. Uncertainty Quantification: Provides valid frequentist coverage for covariance entries through a variance inflation mechanism, supported by theoretical proofs.
4. Theoretical Guarantees:
   • Posterior Contraction: Proves that the posterior concentrates around the true parameters as $n, p_m \to \infty$.
   • Central Limit Theorem (CLT): Establishes asymptotic normality for point estimators of covariance entries.
   • Bernstein-von Mises (BvM): Shows the posterior distribution converges to a Gaussian distribution centered at the point estimator, justifying the use of the inflated variance for credible intervals.

4. Results

Simulation Studies

• Setup: Compared FAMA against MOFA, AJIVE, FABLE (single-view factor analysis on concatenated data), ROTATE, and SVD. Scenarios included balanced and unbalanced dimensions/signal-to-noise ratios.
• Accuracy: FAMA achieved competitive or superior accuracy in estimating intra-view and inter-view covariances, particularly in unbalanced scenarios where other methods failed to capture low-signal views.
• Uncertainty: FAMA's intervals achieved near-perfect (95%) frequentist coverage, whereas competitors like FABLE suffered from significant undercoverage.
• Speed: FAMA was orders of magnitude faster than MCMC-based methods (MOFA) and competitive with fast spectral methods (SVD, AJIVE).

Real Data Application: Cancer Multi-omics

• Dataset: Integrated four views (RNA-seq, SCNV, DNA-methylation, RPPA) from 170 cancer cell samples (TCGA).
• Performance:
  • Out-of-sample Likelihood: FAMA significantly outperformed all competitors in predicting held-out data.
  • Biological Interpretability:
    • Identified distinct gene clusters in RNA-seq (e.g., immune-associated genes vs. epithelial programs).
    • Revealed inverse relationships between DNA methylation and protein expression (e.g., ESR1 gene), consistent with known cancer biology.
    • Captured sparse but meaningful cross-view correlations (e.g., lack of correlation between copy number and RNA levels for many genes).
• Efficiency: FAMA completed the analysis in ~4 seconds, compared to minutes or hours for other methods.

5. Significance

This paper presents a breakthrough in high-dimensional multi-view analysis by bridging the gap between computational speed and statistical rigor.

• Practical Impact: It enables the analysis of massive, heterogeneous datasets (common in genomics and precision medicine) that were previously intractable for full Bayesian inference due to computational costs.
• Theoretical Impact: It provides the first rigorous asymptotic theory (contraction, CLT, BvM) for a spectral-based, surrogate-regression approach to multi-view factor models, validating the use of "plug-in" estimators with corrected uncertainty.
• Methodological Shift: It moves the field away from expensive sampling or brittle variational approximations toward scalable, analytically tractable Bayesian inference that maintains valid uncertainty quantification.