📊 statistics

Practical limitations for real-life application of data fission and data thinning in post-clustering differential analysis

This paper demonstrates that while conditional data fission aims to enable valid post-clustering differential analysis in scRNA-seq by decomposing mixture components, its practical application is fundamentally limited because it requires prior knowledge of the unknown clustering structure to accurately estimate parameters and control Type I error rates.

Original authors: Benjamin Hivert, Denis Agniel, Rodolphe Thiébaut, Boris P. Hejblum

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Benjamin Hivert, Denis Agniel, Rodolphe Thiébaut, Boris P. Hejblum

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Double-Dipping" Problem

Imagine you are a detective trying to solve a mystery. You have a pile of clues (data) from a crime scene.

Step 1: You look at the clues to group them into "suspects." You say, "Okay, these three clues look like they belong to Suspect A, and these three look like Suspect B."
Step 2: You then take the same clues and ask, "Is there a real difference between Suspect A and Suspect B?"

The problem is that you used the clues to create the groups in the first place. If you find a difference in Step 2, it might just be because you forced the groups to look different in Step 1. In statistics, this is called "double-dipping," and it leads to false alarms (thinking you found a difference when there isn't one).

The Proposed Solution: "Data Fission" (Splitting the Clues)

To fix this, scientists proposed a clever trick called Data Fission (or "Data Thinning").

Imagine you have a magical loaf of bread (your data). Instead of using the whole loaf to find the suspects and then the whole loaf to test them, you magically split the loaf into two independent halves:

Loaf A: You use this to find the suspects (Clustering).
Loaf B: You use this to test if the suspects are actually different (Differential Analysis).

Because Loaf A and Loaf B are independent, using Loaf A to find the groups doesn't "cheat" when you test Loaf B. This sounds perfect, right?

The Catch: The "Mixture" Problem

This paper argues that this magical splitting trick doesn't work in the real world for a specific type of data (like single-cell RNA sequencing, which is used to study individual cells).

Here is the analogy:
Imagine your "loaf of bread" isn't a single uniform loaf. It's actually a mixture of two different types of dough baked together: Chocolate and Vanilla.

The Chocolate dough represents one group of cells.
The Vanilla dough represents another group.

The "Data Fission" trick was designed for a loaf that is 100% Vanilla or 100% Chocolate. It doesn't know how to handle a mixed loaf.

Why the Trick Fails

When you try to split the mixed loaf (Chocolate + Vanilla) using the standard rules:

The "Global" Split (Marginal Fission): You try to split the whole loaf based on its average taste.
- Result: You end up with two halves that are both a weird, muddy mix of Chocolate and Vanilla.
- The Consequence: When you use the first half to find the groups, you might think you found a "Chocolate Group" and a "Vanilla Group." But because the second half is still a muddy mix, the test you run on it is contaminated. You get false alarms because the "muddy" second half still secretly remembers the structure of the first half.
The "Perfect" Split (Conditional Fission): To do it right, you would need to know exactly which piece of dough is Chocolate and which is Vanilla before you split them.
- The Catch: If you already knew which piece was Chocolate and which was Vanilla, you wouldn't need to do the detective work (clustering) in the first place!
- The Paradox: To split the data correctly, you need to know the answer. But you need to split the data to find the answer. It's a circular loop.

The Real-World Consequences

The authors ran simulations to prove this. Here is what they found:

The Variance Trap: To split the data correctly, you need to know the "spread" (variance) of the data. In a mixed group, the spread is different for Chocolate and Vanilla. If you guess the spread based on the whole mixed loaf, your guess is wrong.
The Result: Because your guess is wrong, the two halves of the data aren't actually independent. They are still "talking" to each other. This causes the statistical tests to go haywire, producing Type I errors (saying "We found a difference!" when there is actually none).

The Single-Cell RNA Sequencing (scRNA-seq) Example

The paper specifically looked at single-cell RNA sequencing, which is like taking a photo of every cell in your body to see what they are doing.

These cells naturally form groups (like immune cells vs. muscle cells).
Scientists want to use Data Fission to study these groups without cheating.
The Reality: When they tried it on real bone marrow data, the method failed completely. It found "differences" between groups that didn't exist, simply because the method couldn't handle the fact that the data was a complex mixture of different cell types.

The Bottom Line

Data Fission and Data Thinning are like a magic trick that only works on a stage with a single, uniform background.

In the messy, real world where data is a complex mix of different groups (like a chocolate-vanilla swirl):

If you try to split it blindly: You get false results (false positives).
If you try to split it correctly: You need to know the groups beforehand, which defeats the purpose of the analysis.

Conclusion: While the idea of splitting data to avoid cheating is brilliant, the paper concludes that we cannot currently use this method for real-world biological data because we don't have the "secret key" (the true group labels) needed to make the split work. We need new methods that don't rely on knowing the answer before we start the test.

1. Problem Statement

In single-cell RNA sequencing (scRNA-seq) and other high-dimensional data analyses, a common workflow involves a two-step pipeline:

Clustering: Grouping observations (e.g., cells) into homogeneous sub-populations.
Differential Analysis: Testing for differences (e.g., gene expression) between these inferred clusters.

This workflow suffers from "double-dipping" (using the same data for both clustering and testing), which inflates Type I error rates (false positives) because the uncertainty of the clustering step is ignored.

To address this, recent methods like Data Fission (Leiner et al., 2023) and Data Thinning (Neufeld et al., 2024) were proposed. These methods decompose a single observation $X$ into two independent parts, $X^{(1)}$ and $X^{(2)}$ . Theoretically, $X^{(1)}$ is used for clustering, and $X^{(2)}$ is used for differential testing, ensuring independence and valid inference.

The Core Issue: These methods rely on strong parametric assumptions (e.g., Gaussian or Negative Binomial distributions) that assume homogeneity (no underlying clusters). However, real-world data (like scRNA-seq) is inherently heterogeneous and best modeled by mixture distributions. The paper investigates whether Data Fission/Thinning can be practically applied to such mixture data.

2. Methodology and Theoretical Framework

The authors analyze the application of Data Fission/Thinning within the framework of Finite Mixture Models, where data $X_i$ arises from one of $G$ latent clusters with specific parameters ( $\mu_g, \Sigma_g$ ).

A. Marginal vs. Conditional Decomposition

The paper distinguishes between two approaches to decomposition:

Marginal Data Fission/Thinning: Uses global parameters (e.g., the overall variance $\Sigma$ $Σ$ ) estimated from the entire dataset, ignoring latent cluster memberships ( $Z_i$ $Z_{i}$ ).
- Assumption: The data is homogeneous.
- Result: $X^{(1)}$ and $X^{(2)}$ are marginally independent but conditionally dependent given the true cluster $Z_i$ .
Conditional Data Fission/Thinning: Decomposes data using component-specific parameters ( $\Sigma_g$ $Σ_{g}$ ) conditioned on the true cluster membership $Z_i$ $Z_{i}$ .
- Assumption: The true cluster structure is known.
- Result: $X^{(1)}$ and $X^{(2)}$ are conditionally independent given $Z_i$ , and the marginal dependence between them preserves the cluster structure.

B. The Variance Estimation Bias

The authors derive that the independence between $X^{(1)}$ and $X^{(2)}$ strictly depends on using the true scale parameters (variance $\Sigma$ or overdispersion $\theta$ ).

Proposition 1: If a biased estimate $\hat{\Sigma}$ is used instead of the true $\Sigma$ , the covariance between the split parts becomes non-zero: $\text{Cov}(X^{(1)}, X^{(2)}) = \Sigma - \hat{\Sigma}$ .
In mixture models, the global estimate $\hat{\Sigma}$ is a weighted average of component variances plus the variance of the means. This creates a bias relative to the true component-specific variance $\Sigma_g$ .
This bias induces spurious correlations between $X^{(1)}$ and $X^{(2)}$ , causing artifacts in the clustering step ( $X^{(1)}$ ) to "leak" into the testing step ( $X^{(2)}$ ), inflating Type I error.

C. Multivariate Limitations (Negative Binomial)

The paper also highlights that standard Negative Binomial Data Thinning is univariate (applied independently to each gene). It fails to account for multivariate correlations between variables (genes). Even if the overdispersion parameter is perfectly estimated, correlations between genes ( $X_j$ and $X_k$ ) induce dependencies between $X^{(1)}_j$ and $X^{(2)}_k$ , violating the independence assumption required for valid inference.

3. Key Contributions

Theoretical Proof of Incompatibility: The authors prove that applying marginal Data Fission/Thinning to mixture data violates the independence assumption required for post-clustering inference. The method effectively assumes a "global null" of no clusters, which contradicts the reality of clustered data.
Quantification of Type I Error Inflation: They provide an analytical expression linking the bias in variance estimation to the Type I error rate of the downstream t-test. They show that even small biases in estimating the scale parameter lead to significant error inflation.
The "Circularity" Problem: The paper identifies a fundamental logical loop:
- To perform valid Conditional Fission (the only theoretically sound approach for mixtures), one must know the true cluster memberships ( $Z_i$ ) and component-specific parameters.
- However, the goal of the analysis is to discover these clusters.
- Therefore, the method requires the answer (the clusters) to be known before the analysis can begin, rendering it infeasible for real-world discovery.
Multivariate Failure in scRNA-seq: The authors demonstrate that univariate Data Thinning fails in high-dimensional settings (like scRNA-seq) because it ignores inter-gene correlations, leading to massive Type I error inflation even when parameters are correctly specified.

4. Results

The authors validated their theory through simulations and real data analysis (Tabula Sapiens dataset):

Simulation (Gaussian Mixtures):
- Marginal Fission: Led to a severe loss of statistical power and inflated Type I error rates. When a single true cluster was split into two artificial clusters by the algorithm, the marginal fission failed to detect this as a false positive, instead treating the split as real differences.
- Conditional Fission: Achieved perfect Type I error control and high power only when the true cluster labels were provided (Oracle scenario). When labels were estimated, performance degraded.
- Tuning Parameter ( $\tau$ ): The trade-off between clustering accuracy (in $X^{(1)}$ ) and testing power (in $X^{(2)}$ ) is heavily dependent on $\tau$ . Marginal fission cannot balance this effectively due to structural disconnect.
Simulation (Negative Binomial Mixtures):
- Marginal thinning using a global overdispersion estimate resulted in non-uniform p-values and inflated Type I errors.
- Multivariate Correlation: In a setting with correlated variables, Negative Binomial thinning (univariate) showed severe Type I error inflation as correlation strength ( $\rho$ ) increased, whereas multivariate Gaussian fission (which accounts for covariance) maintained control.
Real Data Application (Tabula Sapiens):
- Analyzed a homogeneous bone marrow dataset (no true clusters expected).
- Multivariate Clustering: Applied Negative Binomial thinning followed by k-means and Wilcoxon testing. Result: Type I error rate = 1.0 (100% false positives). The inter-gene correlations preserved after thinning caused the algorithm to find spurious clusters.
- Univariate Clustering: When clustering was done gene-by-gene (ignoring correlations), Type I error was controlled at the nominal level (0.05). This confirmed that the failure was driven by the univariate nature of the thinning method failing to handle multivariate dependencies.

5. Significance and Conclusion

Significance:
The paper delivers a critical "reality check" on Data Fission and Data Thinning. While these methods are mathematically elegant for homogeneous data, the authors demonstrate they are practically unusable for the primary use case they were designed for: post-clustering inference in heterogeneous biological data (like scRNA-seq).

Key Takeaways:

Marginal approaches are invalid: Using global parameters for fission in clustered data induces dependencies that invalidate statistical tests.
Conditional approaches are infeasible: The only valid approach (conditional fission) requires prior knowledge of the clusters, creating a circular dependency that prevents its use in discovery settings.
Multivariate limitations: Univariate thinning methods cannot handle the correlation structures inherent in genomic data, leading to catastrophic false positive rates.

Conclusion:
The authors conclude that Data Fission and Thinning, in their current forms, cannot solve the "double-dipping" problem in real-world post-clustering analysis. They highlight the need for alternative methodologies that can handle unknown class structures and multivariate dependencies without requiring prior knowledge of the very clusters they aim to discover. The paper suggests that future work might explore heteroscedastic models or non-parametric variance estimation, but notes these also face significant practical challenges.