Technical Summary: Synthetic Heterogeneous-Effects LASSO (SHEL)

1. Problem Statement

This paper addresses the challenges of variable selection and post-selection inference in high-dimensional mixed-effects models where data are clustered (e.g., subjects grouped within hospitals or schools). Specifically, the authors investigate the failure of standard marginal-model-based procedures, such as the marginal LASSO, when covariates are heterogeneously distributed across clusters.

The core problem identified is a "target shift" phenomenon. In high-dimensional settings, if covariates vary systematically between clusters, a marginal LASSO estimator (which ignores cluster-specific random effects) tends to use these covariates as sparse proxies for the latent cluster effects (random intercepts). Consequently, the estimator converges to a contaminated target parameter ($\beta^* = \beta_0 + \tilde{\gamma}^*$) rather than the true structural fixed effects ($\beta_0$). This shift induces:

1. False Selections: Variables predictive of cluster heterogeneity are selected even if they have no true structural relationship with the outcome.
2. Invalid Inference: Post-selection inference procedures fail because the selection event is driven by the wrong target parameter.

While existing methods like Generalized Linear Mixed Models (GLMM) with likelihood-based approaches exist, they face computational intractability in high dimensions due to the need for numerical integration over random effects. Conversely, marginal approaches (e.g., Penalized GEE) are computationally feasible but suffer from the target-shift bias described above.

2. Methodology: Synthetic Heterogeneous-Effects LASSO (SHEL)

To resolve the target shift, the authors propose Synthetic Heterogeneous-Effects LASSO (SHEL), a fixed-effects penalized framework.

Core Mechanism

Instead of ignoring cluster heterogeneity or modeling it via random effects integration, SHEL explicitly constructs cluster-level synthetic approximations to the latent heterogeneity.

• Synthetic Approximation: The method identifies covariates with heterogeneous distributions across clusters and constructs a cluster-constant matrix $B$ (e.g., using cluster means of these covariates).
• Joint Penalization: The model estimates the fixed effects ($\beta$) and the coefficients for the synthetic approximations ($\gamma$) simultaneously via a joint penalized objective function:
  $$(\hat{\beta}, \hat{\gamma}) = \arg \min_{\beta, \gamma} \left\{ \frac{1}{2N} \|Y - X\beta - B\gamma\|_2^2 + \lambda_1 \|\beta\|_1 + \lambda_2 \|\gamma\|_1 \right\}$$
  (Extended to Generalized Linear Models as GSHEL using negative log-likelihood).

By explicitly including $B\gamma$ to capture between-cluster variation, the method prevents the structural covariates $X$ from being "contaminated" by the need to proxy for random intercepts. This allows $\hat{\beta}$ to target the true fixed effects $\beta_0$.

Iterative Extension (ISHEL/IGSHEL)

The authors also propose an iterative version (ISHEL) where the synthetic approximation is updated in subsequent steps based on previous estimates. This aims to handle cases where the optimal synthetic approximation is not sparse, though the authors note this complicates post-selection inference due to data-dependent design updates.

3. Theoretical Contributions

The paper establishes rigorous theoretical properties for SHEL and GSHEL in high-dimensional settings ($p, p_0 \gg N$):

1. Consistency under Target Shift: The authors prove that the marginal LASSO converges to the biased target $\beta^*$ when covariates are heterogeneous.
2. Consistency of SHEL: Under a Sparse Synthetic Approximation (SSA) condition (where random intercepts can be approximated by a sparse linear combination of cluster summaries), SHEL consistently estimates $\beta_0$.
3. Robustness to Random Effects: Under a Sparse Synthetic Exogeneity (SSE) condition (where the synthetic approximation captures the dependence between covariates and random effects), SHEL remains consistent even if the approximation error is not asymptotically negligible, provided the error is exogenous.
4. Convergence Rates: The paper derives convergence rates for the estimators, showing they depend on the approximation error $\delta_m$ and the tuning parameters.

4. Post-Selection Inference

A significant contribution is the development of valid inference procedures for the selected variables:

• Polyhedral Selective Inference (for Linear SHEL): Adapting the framework of Lee et al. (2016), the authors derive exact selective inference for linear SHEL. They show that conditioning on the selection event (a polyhedron in data space) yields valid truncated normal pivots. They address the non-i.i.d. nature of clustered errors by incorporating the cluster-level covariance structure into the pivot.
• Debiased Inference (for GSHEL): For generalized linear models, where exact polyhedral inference is difficult, the authors adapt the debiased LASSO approach (Van de Geer et al., 2014). They develop a cluster-level central limit theorem to establish asymptotic normality for the debiased estimator, accounting for within-cluster dependence.

5. Empirical Results

The performance of SHEL/GSHEL was evaluated through extensive simulations and a real-world application.

Simulation Studies

• Variable Selection: In high-dimensional settings with heterogeneous covariates, the marginal LASSO exhibited a high rate of false positives (selecting variables merely as proxies for cluster effects). SHEL and GSHEL maintained low false positive rates comparable to the case where no heterogeneity existed ($p_0=0$).
• Parameter Estimation: SHEL significantly reduced the estimation error ($\|\hat{\beta} - \beta_0\|_1$) and Root Mean Squared Error (RMSE) compared to marginal LASSO as the number of heterogeneous covariates increased.
• Inference: The proposed selective inference (SI) and debiased methods controlled the False Positive Rate (FPR) at nominal levels. In contrast, naive post-selection inference (ignoring the selection bias and clustering) substantially inflated the FPR. The debiased approach was noted to be slightly conservative but stable.

Real Data Application

The method was applied to a longitudinal bulk RNA-seq dataset of enriched blood neutrophils from 306 hospitalized COVID-19 patients.

• Comparison: The marginal LASSO selected 75 genes, while GSHEL selected 37.
• Validation: GSHEL successfully recovered key biological markers identified in the original study (e.g., S100A12, CD177, MCEMP1 as markers of the G-MDSC subtype, and SERPINB2 for longitudinal trajectories).
• Significance: The iterative GSHEL and debiased inference confirmed the statistical significance of these markers, demonstrating the method's ability to filter out noise driven by patient-level heterogeneity and identify true biological signals associated with disease severity.

6. Significance and Claims

The paper claims that SHEL offers a computationally tractable alternative to likelihood-based mixed-effects models for high-dimensional data while avoiding the selection bias inherent in marginal models.

Key claims include:

• Target Preservation: SHEL successfully prevents the "target shift" where covariates are used as proxies for latent cluster effects, ensuring that variable selection targets the structural fixed effects.
• Valid Inference: The proposed inference procedures (polyhedral for linear, debiased for generalized) provide valid statistical guarantees in high-dimensional clustered settings, a gap previously unaddressed by existing methods.
• Practical Utility: The method is demonstrated to be effective in real-world biomedical applications, specifically in identifying robust biomarkers from complex, clustered genomic data.

The authors acknowledge limitations, noting that the iterative version (ISHEL) complicates inference due to adaptive design updates, and that the current framework focuses on heterogeneous intercepts rather than random slopes or nested effects, which are directions for future work.