Technical Summary: Adaptive Partition Factor Analysis

1. Problem Statement

Factor analysis is a standard technique for identifying latent traits influencing multivariate observed variables. However, traditional approaches typically analyze data from a single study, neglecting variations present when data is aggregated from multiple sources. While Multi-Study Factor Analysis (MSFA) has emerged to distinguish between latent traits shared across studies and study-specific components, existing methodologies face significant limitations:

• Rigid Grouping Assumptions: Current MSFA models (e.g., De Vito et al., 2019; Chandra et al., 2025) assume a fixed number of study-specific loading matrices ($\Gamma_s$), one per study. This fails to capture scenarios where:
  • Two or more studies exhibit high homogeneity and share identical latent representations.
  • A single study contains heterogeneous sub-populations with distinct latent structures.
  • Units from different groups share specific characteristics, or units simultaneously exhibit characteristics of multiple groups.
• Information Switching: In existing frameworks, it is often difficult to disentangle signals between shared and specific factors, leading to "information switching" where study-specific factors are incorrectly absorbed into the shared component or vice versa.
• Lack of Flexibility: Previous attempts to address partial sharing (e.g., Grabski et al., 2023) rely on pre-specified group counts or deterministic selection matrices, lacking a unifying framework that spans from classical factor models to complex, adaptive partitions.

2. Methodology: Adaptive Partition Factor Analysis (APAFA)

The authors propose Adaptive Partition Factor Analysis (APAFA), a Bayesian framework that extends MSFA by introducing novel shrinkage priors for latent factors. This approach allows for flexible modeling of varying degrees of group heterogeneity without enforcing a deterministic structure.

2.1 Model Specification

The model represents the observation vector $y_i$ for unit $i$ as:
$$y_i = \Lambda\eta_i + \Gamma\phi_i + \epsilon_i$$
Where:

• $\Lambda\eta_i$ represents the shared latent factors common to all units.
• $\Gamma\phi_i$ represents the specific factors. Unlike standard MSFA, $\Gamma$ concatenates all potential study-specific loading matrices, and $\phi_i$ is an augmented vector of latent factors.
• Crucially, the activation of specific factors is governed by a pattern of zeros and non-zeros in $\phi_i$. The $h$-th element of $\phi_i$ is defined as $\phi_{ih} = \tilde{\phi}_{ih}\psi_{ih}$, where $\psi_{ih}$ is a binary indicator.

2.2 Neural Network Interpretation

The authors interpret the model as a specific single-layer neural network:

• Inputs: Dummy variables $x_i$ indicating study membership (and potentially other covariates $z_i$).
• Hidden Layer: Latent specific factors $\phi_i$.
• Activation: The binary indicators $\psi_{ih}$ are generated via a deterministic activation function $f_h(x_i^\top \beta_h)$, where $\beta_h$ are unknown weights.
• Output: The observed variables $y_i$.
  This formulation allows the latent covariance structure to be informed by independent variables, effectively functioning as a conditional variational autoencoder where the latent structure adapts to the input.

2.3 Prior Specification

The core innovation lies in the use of structured shrinkage priors:

1. Global Shrinkage (CUSP): A cumulative shrinkage process (CUSP) is applied to the global scale parameters $\tau_h^\phi$. This allows the model to learn the effective number of specific factors (neurons in the hidden layer) by shrinking unused factors toward zero.
2. Local Shrinkage (Structured Sparsity): The local scales $\psi_{ih}$ are Bernoulli variables with probabilities determined by a logistic function of the study indicators ($x_i$) and parameters $\beta_h$. This creates a "random partition" of observations based on the sparse structure of $\psi_i$.
3. Factor Loadings: Standard priors are used for $\Lambda$ and $\Gamma$, with increasing shrinkage priors applied to the dimensions of shared factors to determine the number of shared components.

2.4 Identifiability

The paper addresses the "information switching" problem and rotational ambiguity:

• Theorem 1: Establishes conditions under which the model is resistant to information switching. If the specific factor loading matrix $\Gamma$ has full column rank and the number of specific factors $k$ satisfies $k < p(p+1)/2$, the sparsity patterns encoded by $\Psi$ are uniquely identified.
• Theorem 2: Under a "Non-replicable Sparsity Pattern Condition" (where distinct groups have unique activation patterns), the specific factors are fully identifiable up to permutation and sign, provided the shared component is fixed.
• Posterior Computation: Inference is performed via Gibbs sampling.

3. Key Contributions

1. Unified Framework: APAFA bridges the gap between classical factor models and MSFA, accommodating a continuous spectrum of scenarios from no study-specific factors to complex, overlapping sub-populations within and across studies.
2. Adaptive Partitioning: By leveraging informed sparsity, the method automatically discovers the partitioning of units into groups with distinct latent structures without requiring pre-specified group counts or labels.
3. Neural Network Connection: The paper explicitly connects Bayesian factor analysis to neural networks, offering a pathway to integrate subject-specific continuous covariates and generalize the model structure.
4. Identifiability Guarantees: The authors provide theoretical conditions ensuring that specific factors are distinguishable from shared factors, resolving a key limitation in previous MSFA literature.

4. Results

The authors evaluate APAFA through comprehensive simulation studies and real-world applications:

• Simulation Studies:

  • Scenarios: The method was tested against TETRIS (Grabski et al., 2023) across four scenarios: (A) Correctly specified MSFA, (A*) MSFA without labels, (B) Perfectly homogeneous studies (no specific factors), (C) Partially shared factors, and (D) Heterogeneous units within studies.
  • Performance: APAFA demonstrated competitive performance across all scenarios. Notably, in Scenario A* (where group labels were withheld), APAFA successfully recovered the underlying structure, whereas methods relying on fixed study labels struggled. In Scenario B, the shrinkage priors correctly identified the absence of specific factors.
  • Posterior Estimates: Visualizations of posterior estimates showed that APAFA accurately recovered the true sparsity patterns of the latent factors.

• Real Data Applications:

  • Bird Species Co-occurrence: Applied to spatial-temporal survey data, the model successfully separated shared environmental drivers from site-specific deviations, handling the heterogeneity inherent in ecological data.
  • Ovarian Cancer Gene Expression: Applied to high-throughput genomic data, the method distinguished between biological signals shared across cancer subtypes and technical or subtype-specific variations, outperforming standard preprocessing in handling residual heterogeneity.

5. Significance and Claims

The paper claims that APAFA offers a richer insight into multivariate data by moving beyond the rigid assumption that every study has a unique, fixed set of specific factors. By treating the partition of data as a latent, adaptive process driven by structured sparsity, the method:

• Accommodates complex data structures where studies may be homogeneous or contain internal sub-populations.
• Provides a principled Bayesian solution to the "information switching" problem, ensuring that shared and specific signals are correctly attributed.
• Offers a flexible architecture that can be extended to include continuous covariates, effectively generalizing factor analysis through the lens of neural networks.

The authors emphasize that while the method requires suitable regularity conditions for identifiability, it circumvents the well-known issue of rotational ambiguity for specific factors, allowing for straightforward posterior interpretation. The code for reproducing the analyses is made publicly available.