PliableBVS: A flexible Bayesian variable selection method for modeling interactions with mandatory modifying variables

Technical Summary of "PliableBVS: A flexible Bayesian variable selection method for modeling interactions with mandatory modifying variables"

Problem Statement
High-dimensional interaction models are essential for analyzing how large sets of variables (e.g., omics features like gene expression) interact with smaller sets of modifying variables (e.g., clinical covariates). While the "pliable lasso" (Tibshirani and Friedman, 2020) has been proposed as an efficient frequentist method for screening such interactions under an asymmetric weak hierarchical constraint, it operates within a frequentist framework. This limitation prevents the direct provision of uncertainty quantification for parameter estimates or variable inclusion. Furthermore, standard penalized regression methods often struggle to maintain hierarchical constraints between main and interaction effects while providing robust variable selection in high-dimensional settings where the number of predictors far exceeds the number of samples.

Methodology
The authors propose PliableBVS, a Bayesian variable selection approach that extends the pliable lasso framework. The method is designed for both linear and logistic regression settings.

• Model Structure: The model assumes a regression equation where the response $y$ depends on main effects of high-dimensional covariates $X$, mandatory low-dimensional modifying variables $Z$, and interaction terms between $X$ and $Z$. The model enforces an asymmetric weak hierarchy: an interaction effect $\theta_{jk}$ can be non-zero only if the corresponding main effect $\beta_j$ is non-zero.
• Prior Specification:
  • Spike-and-Slab Priors: To induce sparsity, the authors employ zero-inflated mixture priors (spike-and-slab) for both main effects ($\beta_j$) and interaction effects ($\theta_{jk}$).
  • Hierarchical Decision Variables: Two latent decision parameters, $\rho$ and $\zeta$, govern the inclusion of main effects and interaction effects, respectively. These are modeled as probabilities drawn from Beta distributions.
  • Conditional Inclusion: The inclusion probability of an interaction term is conditional on the inclusion of its corresponding main effect. This is achieved by assuming a multivariate zero-inflated mixture prior for $\theta_{jk}$ that is only active if $\beta_j \neq 0$.
  • Shrinkage: The continuous shrinkage component of the priors follows a Laplace distribution (equivalent to the Bayesian lasso variance), while the spike component is a Dirac delta function at zero.
• Inference:
  • Linear Regression: A Gibbs sampler is derived to sample from the joint posterior distribution of all parameters, including coefficients, variance components, and inclusion indicators.
  • Logistic Regression: The model is extended to binary outcomes using the Pólya-gamma augmentation (Polson et al., 2013). This technique introduces latent variables to represent the logistic likelihood as a mixture of Gaussians, allowing the use of the same hierarchical priors and Gibbs sampling structure as in the linear case.
  • Hyperparameter Estimation: The global penalty parameter $\lambda$ is estimated via an empirical Bayes approach using a Monte Carlo EM algorithm.
• Model Selection: The authors utilize the Median Probability Model (MPM) (Barbieri and Berger, 2004) to select variables. A variable is included in the final model if its marginal posterior inclusion probability exceeds 0.5.

Key Contributions

1. Bayesian Extension of Pliable Lasso: The paper introduces a coherent probabilistic framework that mirrors the pliable lasso but provides full posterior distributions, enabling uncertainty quantification for both parameter estimates and variable selection.
2. Structured Sparsity: The method explicitly enforces asymmetric weak hierarchical constraints through a two-layer decision variable structure, ensuring biological and statistical plausibility where interactions cannot exist without their main effects.
3. Algorithmic Implementation: The authors provide a Gibbs sampler for both continuous and binary outcomes, along with an open-source R package (PliableBVS) and a language-agnostic API to facilitate application.

Results
The performance of PliableBVS was evaluated against the frequentist pliable lasso through simulation studies and real-data applications.

• Simulation Studies:
  • Across various scenarios involving different signal-to-noise ratios (SNR) and dimensions (both low and high-dimensional), PliableBVS consistently achieved lower test Mean Squared Error (MSE) and lower false positive rates compared to the pliable lasso.
  • While both methods achieved high sensitivity in identifying true non-zero coefficients, PliableBVS demonstrated superior specificity, reducing the inclusion of spurious variables.
  • In high-dimensional settings with high SNR, PliableBVS showed higher sensitivity than the pliable lasso.
  • The Bayesian approach provided stable, unimodal marginal posterior distributions for coefficients.
• Real Data Applications:
  • Labor Onset Study: Applied to proteomics and metabolomics data to predict the time to spontaneous labor onset. PliableBVS identified biologically meaningful proteins (e.g., Angiopoietin-2, Endostatin, SLPI) and interactions, often selecting fewer features with lower test MSE than the pliable lasso. It successfully identified time-dependent interaction effects for specific proteins.
  • Preeclampsia Study: Applied to a longitudinal proteomics dataset to identify biomarkers for late-onset preeclampsia (LOPE). The model identified known biomarkers (e.g., CRLF1, HTRA1, FLT1) and novel candidates (e.g., FOXM1, CEACAM20). Crucially, it detected gestational age-dependent shifts in the association of certain proteins (e.g., CRLF1) with LOPE risk, a dynamic pattern that might be missed by analyzing time points separately.

Significance and Claims
The paper claims that PliableBVS offers a flexible, interpretable, and computationally efficient tool for high-dimensional regression with structured interactions. By combining the continuous shrinkage of the Bayesian lasso with the discrete variable selection of spike-and-slab priors, the method improves prediction accuracy and reduces false discoveries compared to its frequentist counterpart.

The authors emphasize that the method is particularly valuable for personalized medicine and biomedical research where heterogeneous effects are modulated by a few mandatory variables. However, they maintain a modest stance regarding the biological findings: the identified features and interactions in the real-data applications are presented as hypothesis-generating rather than causal. The authors note that, like other high-dimensional discovery analyses, these results require validation in independent populations before clinical relevance can be established. They also acknowledge that, similar to other lasso-type models, PliableBVS tends to select one variable from a set of highly correlated predictors, suggesting potential future extensions using elastic-net-type priors.