📊 statistics

Zero-inflated modeling with smoothing on counting tensors

This paper proposes a unified probabilistic framework for sparse, zero-inflated count tensors that integrates low-rank CP decomposition with genomic smoothness to jointly model multiway dependencies, cell heterogeneity, and structured variation in data such as single-cell Hi-C.

Original authors: Elena Tuzhilina, Yaoming Zhen

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Elena Tuzhilina, Yaoming Zhen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to reconstruct a massive, complex 3D jigsaw puzzle of a city, but there are two major problems: first, most of the pieces are missing (sparsity), and second, some of the pieces you do have are actually "fake" pieces—they look like they belong, but they are actually just empty cardboard cutouts (zero-inflation).

This paper, "Zero-inflated modeling with smoothing on counting tensors," introduces a new mathematical "super-tool" called ZITS to solve exactly this kind of problem.

Here is the breakdown of how it works using everyday analogies.

1. The Problem: The "Ghost" in the Data

The researchers are looking at single-cell Hi-C data. In plain English: they are trying to map how DNA is folded inside a single cell.

Think of DNA as a very long, tangled piece of yarn. To understand how the cell works, scientists need to know which parts of the yarn are touching each other in 3D space. They use a "counting" method to see how often two points touch.

The two headaches:

The "Dropout" Problem (Zero-Inflation): Imagine you are counting how many times two people shake hands in a crowded room. If you blink, you might miss a handshake. You record a "zero." But was it a zero because they didn't shake hands (a real structural zero), or because you blinked (a technical "fake" zero)?
The "Sparse" Problem: Because cells are tiny, most parts of the DNA never touch. Your data is mostly zeros. It’s like trying to map a city where you only know the location of three streetlights and nothing else.

2. The Solution: The ZITS Framework

The authors propose ZITS, which acts like a high-tech detective with three special skills:

Skill A: The "Truth Detector" (Zero-Inflation Modeling)

Instead of just seeing a "zero" and moving on, ZITS asks: "Is this a real zero or a fake one?"
It uses a probabilistic model (the Zero-Inflated Poisson) to calculate the odds. It’s like a detective looking at a crime scene: "The footprint is missing. Is it because the thief wore no shoes (real zero), or because it rained and washed it away (fake zero)?" ZITS can actually "impute" (guess) what the real number should have been if the "blink" hadn't happened.

Skill B: The "Social Network" Logic (Low-Rank Tensor Decomposition)

ZITS assumes that cells aren't just random; they belong to groups (like families or neighborhoods).
Imagine you are trying to guess the interests of people in a city. If you know that "People in Neighborhood A generally love jazz," and you meet a new person from Neighborhood A, you can make a very educated guess about their music taste. ZITS uses this "group logic" to fill in the blanks across thousands of cells simultaneously.

Skill C: The "Smooth Curve" Rule (Smoothing)

DNA is a continuous string; it isn't a collection of disconnected dots. If point A is touching point B, and point B is touching point C, then point A is likely very close to point C.
ZITS uses "smoothing" to ensure the 3D map looks like a continuous, flowing piece of yarn rather than a jagged, broken mess. It’s like drawing a line through a series of dots—you don't make sharp, impossible zig-zags; you make a smooth, natural curve.

3. Why does this matter? (The Results)

The researchers tested ZITS on real human DNA data and found it was much better than previous methods at:

Seeing through the fog: It correctly identified which "zeros" were just technical errors.
Sorting the crowd: It was much better at grouping similar cells together (clustering).
Building the map: It created much more accurate 3D models of how DNA is organized.

Summary Metaphor

If traditional methods are like trying to look at a blurry, pixelated photo of a forest and guessing where the trees are, ZITS is like using an AI-powered, high-definition scanner that knows how trees grow, knows how forests are structured, and can tell the difference between a real tree and a shadow.

Technical Summary: Zero-Inflated Modeling with Smoothing on Counting Tensors (ZITS)

Authors: Elena Tuzhilina and Yaoming Zhen
Date: April 27, 2026
Field: Statistical Science / Computational Biology / Tensor Decomposition

1. The Problem: Sparsity and Heterogeneity in Multiway Count Data

The paper addresses the challenge of analyzing high-dimensional, sparse count tensors, specifically motivated by single-cell Hi-C data (which measures 3D chromatin organization). These data structures present three simultaneous statistical hurdles:

Extreme Sparsity & Zero-Inflation: Observed zeros in the data are a mixture of structural zeros (true biological absence of contact) and technical zeros (dropout events caused by low sequencing depth or measurement noise).
Cell-to-Cell Heterogeneity: Biological samples are not uniform; cells often belong to distinct subpopulations (clusters) with different structural patterns.
Ordered Smoothness: In genomic data, the indices (loci) are ordered along a chromosome. Neighboring loci exhibit spatial continuity, meaning the underlying signal varies smoothly along the genomic domain.

Existing methods typically address these issues in isolation: zero-inflated models ignore tensor structure, tensor decompositions ignore zero-inflation, and smoothing methods are disconnected from probabilistic tensor frameworks.

2. Methodology: The ZITS Framework

The authors propose the Zero-Inflated Tensor model with Smoothing (ZITS), a unified likelihood-based probabilistic framework. The model is built on three integrated components:

Zero-Inflated Poisson (ZIP) Model: The observed contact counts $C_{i,j,k}$ are modeled as a mixture. A latent Poisson intensity $\lambda_{i,j,k}$ represents the true contact frequency, while a masking probability $p_{i,j,k}$ accounts for the probability that a contact is "dropped" (resulting in a technical zero).
Doubly Low-Rank CP Decomposition: To capture multiway dependence and heterogeneity, the model applies a Canonical Polyadic (CP) decomposition to the transformed intensity ( $\eta = \log \lambda$ ) and masking ( $\theta = \text{logit}(1-p)$ ) tensors. Crucially, it uses a block-structured latent embedding to represent cell clusters, allowing cells within the same cluster to share homogeneous embeddings.
Functional Smoothing via Basis Expansion: To capture the continuous nature of chromatin, the genomic locus embeddings ( $\alpha$ ) are modeled using a linear combination of orthonormal basis functions (e.g., splines). This imposes a smooth curve-like structure on the latent genomic space.

Estimation: The parameters are estimated using alternative gradient descent on the negative log-likelihood. After convergence, a post-hoc clustering step is performed to identify cell populations based on the learned embeddings.

3. Key Contributions

Theoretical Framework: The authors establish the identifiability of the model parameters, proving that the latent cluster memberships and genomic embeddings can be uniquely recovered under mild conditions.
Consistency Guarantees: They derive consistency rates for the maximum likelihood estimator in a high-dimensional regime, showing that the error in estimating the intensity and masking tensors vanishes as the number of cells and loci increases.
Bayes-Optimal Zero Detection: They develop a principled classifier to distinguish structural zeros from technical zeros. By comparing the estimated $p$ and $\lambda$ , the model can decide if a zero is likely a "dropout" and subsequently impute it using the estimated Poisson intensity.

4. Results and Empirical Validation

Simulation Studies: ZITS demonstrates superior performance in recovering latent structures and detecting false zeros compared to standard tensor decomposition and existing imputation methods. The authors also observe a "blessing of sparsity," where higher sparsity actually aids in the accurate estimation of the masking probability $p$ .
Single-cell Hi-C Application: On real human Hi-C data, ZITS significantly outperforms state-of-the-art methods (like HiCImpute and scHiCSRS) in cell-type clustering accuracy (measured by the Adjusted Rand Index).
Imputation Benefit: The study shows that imputing technical zeros using the ZITS framework substantially improves the separability of cell clusters in downstream 3D chromatin organization inference.

5. Significance

This work provides a mathematically rigorous and biologically motivated solution to a complex data science problem. By bridging zero-inflation modeling, tensor factorization, and functional data analysis, ZITS offers a flexible template for any multiway counting data that exhibits sparsity, heterogeneity, and ordered dependencies (e.g., ecological time-series or recommender systems). It moves the field from heuristic imputation toward principled, uncertainty-aware statistical inference.