📊 statistics

Pathway-based Bayesian factor models for 'omics data

The paper introduces BASIL, a scalable Bayesian framework that integrates annotated gene sets into latent variable inference for transcriptomic data, enabling interpretable and reproducible identification of biological pathways without requiring computationally expensive sampling or manual hyperparameter tuning.

Original authors: Lorenzo Mauri, Federica Stolf, Amy H. Herring, Cameron Miller, David B. Dunson

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lorenzo Mauri, Federica Stolf, Amy H. Herring, Cameron Miller, David B. Dunson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding the "Hidden Conductors" in a Noisy Orchestra

Imagine you walk into a massive concert hall where 14,000 musicians (genes) are all playing at once. This is what happens inside your body when you get sick; your cells are shouting out instructions via RNA.

The problem for scientists is that there is too much noise. If you just listen to the whole room, it sounds like chaos. You can't tell if the violin section is playing a sad song or if the drums are just banging randomly.

The Goal: Scientists want to find the "conductors." These are the hidden patterns (biological pathways) that tell groups of musicians to play together. For example, "When we have a fever, the 'Inflammation Orchestra' plays loud, while the 'Sleep Orchestra' goes quiet."

The Old Way: The "Guessing Game" (PLIER)

Previously, scientists used a tool called PLIER to find these conductors. Think of PLIER as a very strict music teacher who only allows musicians to play loud notes.

The Flaw: In biology, sometimes a pathway means "turn this gene off." PLIER can't handle "turning off" (negative numbers) well. It forces everything to be positive, which distorts the music.
The Tuning: PLIER is like a radio with a broken dial. You have to twist the knobs (tune hyperparameters) manually for hours to get a clear signal. If you get it wrong, the music sounds terrible.
The Blind Spot: PLIER gives you a single answer but no idea how confident it is. It's like a weather forecaster saying "It will rain" without telling you if there's a 10% chance or a 90% chance.

The New Way: BASIL (The Smart, Flexible Conductor)

The authors of this paper created a new tool called BASIL (Bayesian Analysis with gene-Sets Informed Latent space).

Think of BASIL as a Smart Music Producer who has a cheat sheet.

1. The Cheat Sheet (Gene Sets)

BASIL doesn't start from scratch. It knows that musicians often play in specific groups (like "The String Section" or "The Brass Section"). In biology, these are called Pathways or Gene Sets.

How it works: BASIL looks at its cheat sheet (a database of known biological pathways) and says, "Okay, these 500 genes usually play together." It uses this knowledge to guide the analysis.
The Twist: Unlike the old tool, BASIL is smart enough to say, "Hey, this group is playing loudly (positive), but that other group is playing silently (negative)." It captures the full complexity of the music.

2. The "Best of Both Worlds" Approach

BASIL realizes that the cheat sheet isn't perfect. Some genes don't fit into any known group.

The Strategy: BASIL splits the job into two parts:
- Part A: Explains the music using the known cheat sheet (Pathways).
- Part B: Creates a "wild card" section for the weird, unknown genes that don't fit the cheat sheet.
The Magic: It automatically figures out how much of the music is explained by the cheat sheet and how much is just random noise or new discoveries. If the cheat sheet is perfect, it ignores the wild cards. If the cheat sheet is useless, it relies on the wild cards.

3. Speed and Confidence (The "Pre-Training" Trick)

Usually, analyzing this much data requires a supercomputer running for days (like a slow, grinding MCMC sampler).

The Analogy: Imagine trying to guess the average height of everyone in a stadium. The old way is to measure every single person one by one.
BASIL's Trick: BASIL takes a quick "snapshot" (a PCA estimate) to get a really good guess first. Then, it uses that guess to do the heavy lifting instantly. It's like taking a high-speed photo and then just refining the details.
Result: It runs in seconds, not days.
Confidence: It also gives you a "confidence interval." Instead of just saying "This pathway is active," it says, "This pathway is active, and I'm 95% sure." This helps doctors know which findings are real and which are just flukes.

What Did They Find? (The Fever Study)

The team tested BASIL on a massive dataset of patients with fevers (from the US and Sri Lanka). They wanted to see what was driving the fever: a virus? Bacteria? Or something else?

The Discovery: BASIL found clear "musical themes."
- Theme 1: "Phosphoinositide Signaling." This is a specific chemical signal that acts like a master switch for cell communication.
- Theme 2: "Interferon-Driven Inflammation." This is the body's "Red Alert" system fighting viruses.
The Comparison: When they compared BASIL to the old tool (PLIER), BASIL's map of the genes was much cleaner. PLIER's map was a tangled mess of weak connections. BASIL's map showed three distinct, clear clusters of genes that made perfect biological sense (like neutrophils, red blood cells, and immune responses).

Why Does This Matter?

Better Diagnosis: By understanding these "hidden conductors," doctors might be able to tell if a fever is caused by a virus or bacteria just by looking at the gene patterns, leading to better antibiotic use.
No More Guessing: It removes the need for scientists to spend hours tweaking settings. It just works.
Trustworthy Results: Because it tells you how confident it is, researchers can stop wasting time on false leads.

In Summary:
BASIL is like upgrading from a blurry, black-and-white photo of a chaotic orchestra to a high-definition, color video where you can see exactly which instruments are playing, how loud they are, and exactly how sure the camera is that it captured the right sound. It uses existing knowledge to guide the search but remains flexible enough to discover new things, all while doing the math incredibly fast.

1. Problem Statement

High-throughput transcriptomic data (e.g., RNA-sequencing) presents a "large $p$ , small $n$ " challenge, where the number of genes ( $p$ ) vastly exceeds the number of samples ( $n$ ). While standard factor analysis and Principal Component Analysis (PCA) are used for dimensionality reduction, they often fail to incorporate existing biological knowledge, resulting in latent factors that are difficult to interpret biologically.

Existing state-of-the-art methods, such as PLIER (Pathway-Level Information Extractor), attempt to integrate pathway knowledge (gene sets) but suffer from critical limitations:

Restrictive Assumptions: PLIER constrains factor loadings to be non-negative, ignoring biologically meaningful negative correlations (e.g., antagonistic pathways).
Lack of Uncertainty Quantification (UQ): PLIER provides only point estimates, failing to quantify uncertainty in the inferred covariance structure or latent factors.
Computational Burden: PLIER requires computationally expensive hyperparameter tuning and often fails to recover the true empirical gene correlation structure accurately.
Overfitting: PLIER tends to overestimate the number of latent factors, leading to less parsimonious models.

2. Methodology: BASIL

The authors propose BASIL (Bayesian Analysis with gene-Sets Informed Latent space), a scalable Bayesian framework designed to integrate annotated gene sets into latent variable inference while maintaining flexibility and computational efficiency.

Model Specification

BASIL models the normalized gene expression matrix $Y$ ( $n \times p$ ) as:
$Y = M\Lambda^\top + E$
where $M$ represents latent factors and $\Lambda$ is the loading matrix. The core innovation lies in the decomposition of the loading matrix $\Lambda$ :
$\Lambda = C\Gamma + \Psi$

$C$ : A binary matrix representing known gene sets (pathways, GO terms).
$\Gamma$ : A structured matrix linking latent factors to known gene sets.
$\Psi$ : An unstructured matrix capturing variation not explained by known gene sets (novel pathways or noise).
Orthogonality: The model enforces $\Psi$ to lie in the null space of $C$ ( $\Psi \in \mathcal{N}(C)$ ), ensuring identifiability and separating structured from unstructured variation.

Priors and Shrinkage

Structured Shrinkage: The elements of $\Gamma$ and $\Psi$ are assigned Gaussian priors with distinct variance parameters ( $\tau^2_\Gamma$ and $\tau^2_\Psi$ ). This allows the model to adaptively shrink the unstructured component if the data is well-explained by known pathways, or vice versa.
Empirical Bayes: Hyperparameters (shrinkage coefficients) are selected automatically via an Empirical Bayes procedure, eliminating the need for manual tuning.
Factor Selection: The number of latent factors ( $k$ ) is selected automatically using a Joint Likelihood-based Information Criterion (JIC).

Scalable Inference

To avoid the computational prohibitions of Markov Chain Monte Carlo (MCMC) in high dimensions, BASIL employs a pre-training approach:

Pre-estimation: Latent factors $M$ are initially estimated using a PCA-based spectral decomposition of $Y$ .
Conditional Posterior: The posterior distribution of the loadings ( $\Lambda$ ) is approximated conditionally on the pre-estimated $M$ . This transforms the problem into a set of independent, conjugate regression tasks.
Coverage Correction: To correct for the mild undercoverage inherent in pre-training, a variance inflation factor ( $\rho^2$ ) is applied to the posterior variance, ensuring valid frequentist coverage for credible intervals.

3. Key Contributions

Biological Interpretability: By explicitly modeling the contribution of known gene sets, BASIL produces latent factors that are directly interpretable as biological pathways.
Flexibility: Unlike PLIER, BASIL allows for negative loadings, capturing antagonistic biological relationships (e.g., immune activation vs. transport).
Uncertainty Quantification: The framework provides rigorous, calibrated credible intervals for the covariance matrix and latent factors, crucial for distinguishing signal from noise in high-dimensional data.
Computational Efficiency: The pre-training strategy enables analysis of datasets with tens of thousands of genes in seconds, avoiding the slow convergence of MCMC.
Adaptive Sparsity: The model automatically determines the relevance of known pathways versus unknown components, collapsing to simpler models when appropriate.

4. Results

Simulation Studies

Accuracy: BASIL outperformed PLIER and ROTATE in recovering the true low-rank structure of the gene covariance matrix across both "high ontology signal" and "low ontology signal" scenarios.
Factor Selection: BASIL consistently recovered the true number of latent factors ( $k$ ), whereas PLIER significantly overestimated it (e.g., estimating 60+ factors when the truth was 10).
Speed: BASIL was approximately 30 times faster than PLIER.
Uncertainty: BASIL achieved ~93-95% frequentist coverage for 95% credible intervals, whereas competitors offered no UQ.

Application 1: Whole-Blood RNA-seq (Mao et al., 2019)

Dataset: 36 samples, ~5,892 genes.
Findings: BASIL identified three distinct, biologically interpretable gene clusters (neutrophil granules, azurophilic granules, and erythrocytes).
Comparison: PLIER produced a dense network with many spurious weak correlations. BASIL produced a sparse network by zeroing out entries where the 95% credible interval included zero, effectively separating signal from noise and enhancing interpretability.

Application 2: Global Fever Transcriptomic Cohort (Ko et al., 2023)

Dataset: 291 patients (bacterial, viral, non-infectious), 14,386 genes.
Findings:
- BASIL identified 19 interpretable factors.
- Key Drivers: Phosphoinositide signaling and interferon-driven inflammation were identified as major drivers of gene expression variability.
- Specific Pathways: Factors captured trade-offs between innate immune signatures (TLR4 binding) and ion transport, as well as mitochondrial stress responses.
- Negative Correlations: The model successfully captured opposing relationships (e.g., membrane transporters vs. phosphatases) that would be missed by non-negative methods like PLIER.
- Gene Specificity: The model correctly attributed the variance of immune markers (e.g., CXCL10, IFI27) to known interferon pathways, while identifying genes like OTOF (auditory function) as having variance driven by unknown components.

5. Significance

BASIL represents a significant advancement in the analysis of high-dimensional 'omics data by bridging the gap between statistical rigor and biological interpretability.

Reproducibility: The automated selection of hyperparameters and the number of factors makes the method robust and user-friendly, reducing the "black box" nature of current factor models.
Clinical Utility: By providing uncertainty quantification, BASIL allows researchers to distinguish robust biological signals from noise, which is critical for identifying biomarkers in complex diseases like sepsis or fever.
Scalability: The method makes pathway-informed Bayesian analysis feasible for large-scale transcriptomic studies without requiring massive computational resources.
Future Directions: The framework is designed to be extensible to single-cell RNA-seq, multi-omics integration, and clustered data settings, offering a versatile tool for precision medicine.