📊 statistics

Nested Atoms Model with Application to Clustering Big Population-Scale Single-Cell Data

This paper proposes the Nested Atoms Model (NAM), a scalable Bayesian nonparametric approach that jointly clusters individuals and single cells by integrating group-level genotypes with observation-level gene expressions, thereby enabling the discovery of biologically meaningful cell-type profiles influenced by genetic variation in large-scale datasets like OneK1K.

Original authors: Arhit Chakrabarti, Yang Ni, Yuchao Jiang, Bani K. Mallick

Published 2026-04-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Arhit Chakrabarti, Yang Ni, Yuchao Jiang, Bani K. Mallick

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to organize a massive library, but this isn't just any library. It's a library where every single book (a cell) belongs to a specific person (an individual), and every person has their own unique DNA blueprint (genetics).

In the past, statisticians had two ways to organize this library:

The "Book-Only" Method: They looked at the books and grouped them by story type (e.g., all mystery novels together, all sci-fi together). But they completely ignored who owned the books.
The "Owner-Only" Method: They looked at the owners and grouped them by their DNA. But they ignored what books were actually on the shelves.

The problem is that in real life, both matter. People with similar DNA might have similar "bookshelves" (cell types), but sometimes, two people with very different DNA might end up with surprisingly similar bookshelves because of how they live or their environment. Existing methods were bad at handling this "nested" mess where you have to sort both the owners and their books simultaneously.

The Solution: The "Nested Atoms Model" (NAM)

The authors of this paper, led by Arhit Chakrabarti and colleagues, invented a new tool called the Nested Atoms Model (NAM).

Here is how it works, using a simple analogy:

1. The "DNA Fingerprint" and the "Bookshelf"

Think of every person in the study as a House.

The House's DNA (Group-Level): This is the blueprint of the house. It tells you if the house is a Victorian, a Modern, or a Cottage. In the study, this is the person's genetic data (SNPs).
The Books on the Shelf (Observation-Level): Inside each house, there are thousands of books (cells). These books tell you what the house is doing right now. Are there medical textbooks? (Immune cells fighting infection). Are there cookbooks? (Digestive cells).

2. The Old Way vs. The New Way

Old Methods (CAM, nDP): Imagine a librarian who tries to sort the books. They look at the books and say, "Oh, these two houses both have a lot of medical textbooks, so they must be the same type of house." But they ignore the fact that one house is a Victorian and the other is a Modern apartment. They miss the big picture.
The New Method (NAM): This librarian is smarter. They look at both the blueprint of the house and the books on the shelf.
- If two houses have similar blueprints (DNA) and similar books, NAM groups them together.
- If two houses have different blueprints but surprisingly similar books, NAM notices that too and groups them based on the books.
- Crucially, it realizes that even if House A and House B are different types, they might share a "common library" of book types (cell types) that appear in both, just in different quantities.

3. The "Atom" Concept

The name "Nested Atoms" sounds fancy, but think of "Atoms" as the building blocks of identity.

In the old models, the building blocks were rigid. If two houses shared one block, they were forced to be the exact same type of house.
NAM uses "Common Atoms." Imagine a set of universal LEGO bricks. Every house uses these same bricks to build their rooms. NAM figures out which bricks are used most often in House A versus House B. This allows it to say, "These two houses are built differently, but they both use the same 'Red Brick' (a specific cell type) in their kitchen."

Why Does This Matter? (The OneK1K Story)

The researchers tested this on a massive dataset called OneK1K.

The Scale: They looked at 1.27 million cells from 982 different people. That is like trying to sort a billion books from a thousand different families all at once.
The Challenge: Doing this with old computers and old math would take forever or give messy results.
The Breakthrough: The authors built a "fast-forward" engine (called Variational Inference) that lets the computer sort this data quickly without getting stuck.

What Did They Find?

When they applied NAM to the real data, it worked like magic:

Genetic Clusters: It grouped people with similar DNA together.
Cell Clusters: It identified specific types of immune cells (like B-cells and T-cells) across all these people.
The "Aha!" Moment: They found that people with similar DNA tended to have similar mixes of immune cells. But more importantly, NAM could spot when people with different DNA still had similar cell mixes, which tells scientists that environment or other factors are at play.

They also found specific "marker genes" (like a unique barcode on a book) that confirmed these groups were real biological types, not just random math errors. For example, they found a gene called MS4A1 that was a dead giveaway for B-cells, appearing exactly where the model predicted it would.

The Bottom Line

This paper is about building a better sorting machine for the most complex data we have: human biology.

Before this, if you wanted to understand how our genes affect our cells, you had to look at the genes and the cells separately. This new model, NAM, lets us look at them together. It's like finally having a map that shows not just where the houses are, but exactly what's inside them, and how the architecture of the house influences the furniture inside.

This helps scientists understand diseases better, figure out why some people get sick and others don't, and eventually, design better treatments tailored to both your DNA and your specific cell behavior.

1. Problem Statement

The paper addresses the challenge of clustering nested (hierarchical) data where observations are grouped, and variables exist at both the group level and the observation level.

Context: The motivating application is the OneK1K study, a large-scale single-cell RNA-sequencing (scRNA-seq) dataset containing 1.27 million cells from 982 individuals.
Data Structure:
- Observation-level variables ( $y_{ji}$ ): Cell-specific gene expression data (scRNA-seq).
- Group-level variables ( $x_j$ ): Individual-specific genotype data (SNPs).
The Gap: Existing Bayesian nonparametric clustering methods (e.g., Hierarchical Dirichlet Process, Nested Dirichlet Process, Common Atoms Model) are designed for grouped data but fail to incorporate group-level variables. Consequently, they cannot jointly cluster individuals based on their genetic profiles while simultaneously clustering cells based on gene expression, nor can they account for how genetic variations influence cell-type heterogeneity.

2. Methodology: The Nested Atoms Model (NAM)

The authors propose NAM, a new Bayesian nonparametric framework designed specifically for nested data.

Model Formulation

NAM models the joint distribution of group-level variables ( $x_j$ ) and observation-level variables ( $y_{ji}$ ) using a two-layered clustering structure:

Group Clusters (GCs): Individuals ( $j$ ) are clustered based on $x_j$ (genotypes).
Observational Clusters (OCs): Cells ( $i$ within group $j$ ) are clustered based on $y_{ji}$ (gene expression).

Key Mathematical Components:

Random Measures: The model assumes a random probability measure $Q$ $Q$ over the space of distributions.
- $G_j = (G^x_j, G^y_j)$ represents the distribution for group $j$ .
- $G^x_j$ (for genotypes) is a point mass on an atom $\theta^x_k$ .
- $G^y_j$ (for gene expression) is a mixture of common atoms $\theta^y_l$ .
Coupling: The atoms $\theta^x_k$ (genetic) are coupled with the mixture weights of the cell clusters ( $G^y_j$ ). This allows the genetic profile of an individual to influence the probability of their cells belonging to specific cell-type clusters.
Common Atoms: The model utilizes a set of common atoms ( $\theta^y_l$ ) shared across all groups, ensuring that cell types identified in one individual are comparable to those in another (e.g., "T-cells" in Individual A are the same cluster as "T-cells" in Individual B).
Prior Structure: The model uses stick-breaking constructions (GEM distributions) for weights $\pi$ (group level) and $\omega$ (observation level). Crucially, the prior correlation structure depends on the similarity of the group-level variables ( $x_j$ ), allowing individuals with similar genotypes to have stronger prior correlation in their cell-type distributions.

Scalable Inference

Given the massive scale of the data (1.27 million cells), standard Markov Chain Monte Carlo (MCMC) is computationally infeasible.

Variational Inference (VI): The authors developed a fast Variational Bayesian inference algorithm.
Truncation: They employ truncated variational families (limiting the number of potential clusters $K$ and $L$ ) to handle the nonparametric nature of the model.
Optimization: The algorithm uses Coordinate Ascent Variational Inference (CAVI) to maximize the Evidence Lower Bound (ELBO).
Theoretical Guarantees: The paper provides bounds showing that the error between the truncated posterior and the true posterior decays exponentially with the truncation levels $K$ and $L$ .

3. Key Contributions

Novel Model (NAM): Introduction of the first Bayesian nonparametric model capable of simultaneous two-layered clustering while explicitly incorporating group-level covariates (genotypes) to inform both group and observation clustering.
Theoretical Properties: Derivation of the prior mean, variance, and co-clustering probabilities. The authors show that NAM's prior correlation structure is non-stationary and depends on the "odds-ratio" of the base measures, allowing for flexible dependence between genetic similarity and cell-type composition.
Scalable Algorithm: Development of a variational inference algorithm that scales to millions of observations, overcoming the limitations of MCMC-based approaches for large population-scale datasets.
Robustness: Demonstration that the model remains robust even when group-level variables are partially omitted or when data deviates from Gaussian assumptions (e.g., t-distributions).

4. Results

Simulation Studies

Superior Group Clustering: NAM significantly outperformed existing methods (CAM, fiSAN) in identifying Group Clusters (GCs), especially when group-level variables were high-dimensional. CAM and fiSAN, which ignore group-level variables, performed poorly on GC accuracy (ARI $\approx$ 0.49 vs. NAM's 1.00 in some scenarios).
Observation Clustering: NAM achieved comparable accuracy to CAM and fiSAN in identifying Observation Clusters (OCs), proving that incorporating group data does not degrade cell-level clustering.
Scalability: The VI algorithm demonstrated linear scaling with respect to the number of groups ( $J$ ) and observations per group ( $n_j$ ). Runtime remained manageable (median < 2 mins for small dimensions, max < 35 mins for large datasets).
Partial Omission: Even when 20% of group-level variables were omitted, NAM still outperformed methods that completely ignored group data.

Real Data Application (OneK1K)

Clustering Output: NAM identified 6 Group Clusters (individuals) and 14 Observation Clusters (cell types).
Biological Validity:
- Individuals within the same GC showed highly similar cell-type compositions (e.g., similar proportions of B-cells, T-cells).
- Differential Expression: Analysis revealed that individuals in different GCs, even within the same cell cluster (OC), exhibited distinct gene expression patterns. For example, the gene MS4A1 (a B-cell marker) showed differential expression between GCs, confirming that genetic background influences cell states.
- Gene Ontology (GO) Enrichment: Automated GO analysis confirmed that the inferred cell clusters corresponded to known biological identities (e.g., OC 2 enriched for B-cell receptor signaling; OC 40 enriched for T-cell signaling).
Comparison with fiSAN: When applied without SNP data (using fiSAN), the model produced an excessive number of clusters (37 GCs) and failed to group genetically similar individuals with similar cell profiles correctly. NAM's integration of SNPs led to more biologically coherent and parsimonious clustering.

5. Significance

Disentangling Heterogeneity: NAM provides a rigorous statistical framework to disentangle the complex interplay between genetic variation (inter-individual) and cellular heterogeneity (intra-individual).
Population Genomics: It enables the identification of "genetically governed" cell types, revealing how specific genetic backgrounds drive differences in immune cell profiles across a population.
Scalability for Big Data: The variational inference approach makes Bayesian nonparametric clustering feasible for datasets with millions of cells, a critical step for the future of single-cell population genomics.
Biological Insight: The application to OneK1K demonstrates that ignoring genotype data leads to suboptimal clustering, whereas NAM successfully uncovers biologically meaningful subgroups of individuals and their associated cellular states, offering a pathway for automated, large-scale cell-type annotation.