Beyond Local Independence: High-Dimensional Latent Class Graphical Models with Shared Block Structure

Technical Summary: Beyond Local Independence: High-Dimensional Latent Class Graphical Models with Shared Block Structure

Problem Statement
Latent class models (LCMs) are standard tools for analyzing multivariate categorical data from heterogeneous populations. However, traditional LCMs rely on the assumption of local independence: conditional on the latent class, observed responses are mutually independent. This assumption is frequently violated in modern high-dimensional applications, such as public opinion surveys (where items on similar topics remain dependent) and genetic data (where nearby SNPs exhibit linkage disequilibrium). Ignoring such local dependence distorts parameter estimates and latent class assignments.

Existing approaches to relax local independence are typically limited to low-dimensional settings, rely on prespecified or unstructured dependence patterns, or lack statistical guarantees. Conversely, high-dimensional graphical models can learn dependence structures but generally fail to account for unknown heterogeneous latent classes. There is a gap in principled statistical methodology that can jointly learn latent heterogeneity and structured local dependence from high-dimensional ordinal responses.

Methodology
The authors propose a high-dimensional latent class graphical model with a shared block structure. The model assumes that conditional on a latent class $Z_i = k$, ordinal responses are generated via a latent Gaussian copula. The key innovation is that the covariance matrix $\Sigma_k$ of the latent Gaussian vector is block-diagonal after a shared permutation of variables across all $K$ latent classes.

• Shared Block Partition: Variables are partitioned into $L$ blocks ($V_1, \dots, V_L$) that are shared across all latent classes. This captures interpretable groups of locally dependent variables (e.g., survey items on a specific topic).
• Class-Specific Dependence: Within each block, the conditional dependence structure (encoded by the precision matrix $\Omega_k$) is allowed to vary across latent classes, capturing class-specific heterogeneity.
• Three-Step Estimation Pipeline: The authors develop a scalable, unsupervised three-step estimator:
  1. Spectral Clustering: The latent class labels $Z$ are recovered by applying spectral clustering to a "flattened" binary response matrix. This leverages the low-rank structure of the data to handle the ordinal nature of the responses.
  2. Block Structure Recovery: Within each estimated latent class, pairwise polychoric correlations are estimated to recover the class-specific covariance matrices $\Sigma_k$. These are aggregated across classes to identify the shared block partition. The authors propose using community detection algorithms (specifically Leiden) on a continuous aggregation matrix of these correlations to robustly recover the blocks.
  3. Precision Matrix Estimation: Once the blocks are identified, sparse precision matrices $\Omega_k$ are estimated within each block using standard graphical model estimators (e.g., Graphical Lasso or CLIME), conditioned on the known block structure.

Key Contributions

1. Model Formulation: A new model that bridges the gap between classical locally independent LCMs and unrestricted mixtures of graphical models. It imposes a shared block structure to ensure interpretability and parsimony while allowing flexible, class-specific dependence within blocks.
2. Scalable Algorithm: A computationally efficient three-step pipeline that avoids the intractable full likelihood optimization required by unrestricted locally dependent LCMs. The method scales to high dimensions ($J \gg N$).
3. Theoretical Guarantees: The paper establishes finite-sample error bounds for all stages of the pipeline:
   • Clustering accuracy (Proposition 1).
   • Entry-wise consistency for covariance matrix estimation and block recovery (Propositions 3 and 4).
   • Support recovery and sign consistency for precision matrix estimation (Proposition 5).
   • End-to-end consistency: Theorem 1 demonstrates that all model components (latent classes, block partition, and precision matrices) are consistently estimated under high-dimensional scaling.

Results

• Simulations: Extensive simulations across varying dimensions ($J \in \{100, 500\}$), numbers of latent classes ($K$), and block structures ($L$) show that the method accurately recovers latent classes, shared block partitions, and class-specific precision matrices. The method outperforms joint graphical lasso (JGL) baselines, which struggle to recover the block structure without oracle tuning. Ablation studies confirm that both the clustering step (Step 1) and the block recovery step (Step 2) are crucial; omitting either leads to significant performance degradation.
• Real Data Applications:
  • ANES Survey Data: Applied to 2022 American National Election Studies data ($N=1511, J=144$). The method identified three latent classes corresponding to Republican, Democrat, and Independent voters. It recovered interpretable blocks (e.g., "Racism," "Political Engagement") and revealed class-specific dependence patterns, such as differing conditional dependencies regarding racial advantages across ideological groups.
  • HapMap3 Genetics Data: Applied to SNP genotype data ($N=479, J=500$) from four subpopulations. The method successfully recovered the population structure and identified blocks of dependent SNPs (linkage disequilibrium blocks) while accounting for the latent subpopulations. Omitting the clustering step resulted in inflated cross-block correlations, demonstrating the necessity of accounting for heterogeneity.

Significance and Claims
The paper claims to provide a principled, interpretable, and theoretically grounded methodology for relaxing the local independence assumption in high-dimensional latent class analysis. By introducing a shared block structure, the method balances the need for parsimony with the flexibility to model complex dependence. The authors emphasize that their approach offers end-to-end theoretical guarantees for all model components, a feature often missing in existing high-dimensional mixture models. The method is presented as a practical tool for uncovering interpretable local dependence structures in the presence of latent heterogeneity, with demonstrated utility in both social science and genetics. The authors note that the R code is publicly available to facilitate reproducibility.