Benchmarking computational decontamination of ambient RNA

This study rigorously benchmarks seven state-of-the-art computational methods for removing ambient RNA from single-cell and single-nucleus RNA sequencing data, revealing that while no single method excels across all scenarios, CellBender, DecontX, and SoupX generally demonstrate superior performance.

The Gist

Imagine you are trying to take a crystal-clear, high-resolution photo of a specific group of people at a crowded party. You want to see exactly what each person is wearing and saying. But, there's a problem: the room is filled with floating confetti, glitter, and stray pieces of paper that have blown off other tables. Some of this "ambient" debris lands on your camera lens or gets mixed in with your photo.

In the world of biology, scientists use a technology called single-cell RNA sequencing to take "photos" of individual cells to understand how our bodies work. However, just like that party photo, the data gets contaminated with ambient RNA. This is genetic material (RNA) that leaks out of broken or dying cells during the experiment and floats around, getting accidentally captured along with the healthy cells.

This paper is a big, rigorous taste test (a benchmark) to see which digital "cleaning tools" work best at removing that floating debris without accidentally wiping out the actual people in the photo.

The Problem: The "Noisy Room"

When scientists prepare cells for sequencing, some cells inevitably burst open. Their genetic contents spill out into the solution, creating a "soup" of RNA. When the machine reads the data, it can't always tell if a piece of RNA came from the cell it's supposed to be reading or if it just drifted in from a neighbor.

If you don't clean this up, your results are wrong. You might think a healthy cell is sick, or you might miss rare cell types entirely because they are hidden under the noise.

The Contestants: 7 Digital Janitors

The authors tested 7 different computer programs (algorithms) designed to act as digital janitors. Their job is to look at the messy data, figure out what is "real" cell signal and what is "ambient" noise, and subtract the noise.

The tools tested were:

1. CellBender
2. DecontX
3. SoupX
4. FastCAR
5. scAR
6. scCDC
7. CellClear

The Test Drive: How They Were Judged

To see who was the best, the researchers didn't just guess. They created three types of "test rooms":

1. The "Fake" Room (Simulated Data): They built a computer simulation where they knew exactly how much noise was added. This was like a controlled lab where they knew the ground truth.
2. The "Species Mix" (Real Life): They mixed human cells with mouse cells. Since human and mouse genes are different, any human gene found inside a mouse cell was definitely "noise" (ambient RNA). This gave them a real-world ground truth.
3. The "Clean Room" (Negative Control): They took a dataset that was already known to be clean (Smart-seq2) to see if the tools would accidentally create problems where none existed (over-cleaning).

They judged the tools on two main criteria:

• The Vacuum Test: Did it suck up all the floating confetti (ambient RNA)?
• The Preservation Test: Did it accidentally vacuum up the people's clothes (real cell data) along with the confetti?

The Results: Who Won?

Here is the verdict, translated into plain English:

• There is no "Perfect" Tool: Just like there is no single vacuum cleaner that works best on every type of carpet, no single tool won every category.

• The Top Performers:

  • CellBender: The heavy-duty industrial vacuum. It's incredibly good at removing noise and keeping the real data safe, but it requires a lot of power (computing resources) and takes a long time to run. It needs a special "GPU" (a powerful graphics card) to work.
  • DecontX: The reliable, all-around cleaner. It does a great job, though it removes slightly less noise than CellBender. It's a bit more flexible.
  • SoupX: The lightweight, portable vacuum. It's the only one that works well if you only have the "filtered" data (the cleaned-up list of cells) and don't have access to the raw, messy data. It's fast and doesn't need a supercomputer.

• The Cautionary Tales:

  • scAR: This tool was very aggressive. It cleaned the room too well, often throwing away the furniture (real data) along with the dust. In fact, it sometimes deleted so much data that entire cells disappeared from the analysis.
  • FastCAR & scCDC: These were a bit too gentle. They mostly cleaned up the big, obvious dust bunnies (highly expressed genes) but left the smaller, tricky dust behind.

The Big Takeaway

The authors conclude that you can't just blindly run a cleaning tool on every dataset.

• If you have a powerful computer and raw data: Use CellBender. It's the most accurate.
• If you want a balance of speed and accuracy: Use DecontX.
• If you only have filtered data or a weak computer: Use SoupX.

The Golden Rule: Before you start cleaning, check if the room is actually dirty! If you apply these tools to a dataset that is already clean, you might accidentally ruin your data. The best approach is to know your experiment: if you are working with "nuclei" (the core of the cell) rather than whole cells, expect more noise and choose a stronger tool.

In short, this paper gives scientists a user manual for choosing the right digital broom for their specific mess, ensuring that the biological discoveries they make are real and not just an illusion caused by floating genetic dust.

Technical Summary

1. Problem Statement

Single-cell and single-nucleus RNA sequencing (scRNA-seq and snRNA-seq) have revolutionized genomics by enabling high-resolution cellular profiling. However, a pervasive technical artifact known as ambient RNA (extraneous RNA molecules released from lysed cells during sample preparation) contaminates these datasets.

• Impact: Ambient RNA leads to spurious gene expression signals, impairs accurate cell type classification, and confounds downstream analyses (e.g., differential expression, trajectory inference).
• Current Gap: While numerous computational tools exist to remove ambient RNA, there has been no comprehensive, independent benchmark evaluating these methods across diverse data types (synthetic, real-world species/strain mixing, and negative controls). Researchers lack clear guidance on which tool to select based on data availability, computational resources, and contamination levels.

2. Methodology

The authors conducted a rigorous benchmark of seven state-of-the-art computational decontamination methods:

1. CellBender
2. DecontX
3. FastCAR
4. scAR
5. scCDC
6. SoupX
7. CellClear

Datasets Used:
To establish "ground truth" for evaluation, the study utilized four categories of datasets:

• Species-Mixing Experiments (3 datasets): Mixtures of human and mouse cells (low, medium, and high complexity). Ground truth was defined by reads mapping to the "minor" species within cells assigned to the "major" species.
• Genotype-Mixing Experiment: Mixtures of kidney cells from three different mouse strains. Ground truth was established using strain-specific SNPs.
• Synthetic Datasets: Simulated data generated by sampling from negative binomial distributions of real cell lines with defined ambient RNA fractions.
• Negative Controls: Smart-seq2 datasets (human islets) known to lack ambient contamination, used to test for over-correction (false positives).

Evaluation Metrics:
The authors developed a multi-faceted evaluation framework:

• Ambient Removal Score (ARS): Measures the fraction of ambient RNA successfully removed.
• Endogenous Retain Score (ERS): Measures the fraction of true biological signal preserved.
• Biological Interpretation Impact: Evaluated via:
  • Clustering Quality: Adjusted Rand Index (ARI) and silhouette width.
  • Cellular Neighborhoods: Purity and Local Inverse Simpson Index (LISI).
  • Marker Gene Quality: Entropy, Moran's I (spatial autocorrelation), and Gini coefficient.
  • Label Transfer: Accuracy of transferring cell type labels using Azimuth.
  • Batch Integration: Performance of Harmony integration.

3. Key Contributions

• Comprehensive Benchmark: The first independent, large-scale comparison of 7 major ambient RNA removal tools across 97,357 cells/nuclei in 49 real samples and various synthetic scenarios.
• Characterization of Ambient RNA: Detailed analysis revealing that ambient RNA is not random; it is strongly correlated with highly expressed, ubiquitously expressed genes (mature cytoplasmic mRNAs) and is more prevalent in single-nucleus (snRNA-seq) than single-cell (scRNA-seq) data.
• Input Requirement Analysis: Clarified which tools require full unfiltered count matrices (e.g., CellBender) versus those that can operate on filtered matrices (e.g., SoupX, DecontX), a critical factor for public datasets where raw data is often unavailable.
• Over-correction Assessment: Systematically evaluated the risk of removing biological signal in datasets where no contamination exists, highlighting the danger of applying these tools blindly.

4. Key Results

Performance Overview:

• No Universal Best: No single method outperformed all others across every metric and dataset type.
• Top Performers: CellBender, DecontX, and SoupX generally performed best.
  • CellBender: Achieved the highest overall score, excelling in both ambient removal and endogenous retention. However, it is computationally expensive (requires GPU, high memory) and requires unfiltered data.
  • DecontX (Full Mode): Strong performance, particularly in improving biological interpretation (clustering and label transfer), though it removes slightly less ambient RNA than CellBender.
  • SoupX (Reduced Mode): The only top-tier tool that works effectively on filtered count matrices (no access to raw data). It showed a slight bias toward removing ambient RNA from highly expressed genes but preserved endogenous signals well.

Method-Specific Findings:

• scAR: Highly effective at removing ambient RNA but aggressively removes endogenous signal, particularly from lowly expressed genes. It caused significant distortion in negative control datasets (over-correction) and degraded batch integration.
• CellClear & FastCAR/scCDC: Showed specific biases. CellClear tended to remove ambient RNA from lowly expressed genes while altering endogenous profiles significantly. scCDC and FastCAR focused on highly expressed genes.
• Over-correction Risk: In negative control datasets (no ambient RNA), most methods removed some endogenous signal. scAR and CellBender (in the absence of contamination) were most prone to over-correction, potentially distorting biological conclusions.

Ambient RNA Characteristics:

• Ambient load varies significantly between cells (stochastic) but is generally higher in snRNA-seq (11.8%) than scRNA-seq (7.8%).
• It is strongly correlated with gene expression frequency in empty droplets and the fraction of exonic reads.
• Standard quality control metrics (e.g., mitochondrial fraction) are poor predictors of ambient load at the single-cell level.

5. Significance and Recommendations

This study provides a critical decision framework for the single-cell genomics community:

1. For Contaminated Datasets with Full Data Access:

   • Recommendation: Use CellBender (if GPU/resources are available) or DecontX (Full Mode).
   • Reasoning: Best balance of removing contamination while preserving biological signal and improving downstream clustering/labeling.

2. For Contaminated Datasets with Only Filtered Matrices (Public Data):

   • Recommendation: Use SoupX (Reduced Mode).
   • Reasoning: It is the only robust method that does not require the unfiltered count matrix.

3. For Datasets with Low/Unknown Contamination:

   • Recommendation: Use SoupX (Reduced Mode) or exercise caution.
   • Reasoning: Tools like CellBender and scAR carry a higher risk of over-correction (removing real biological signal) if contamination is minimal.

Conclusion:
The paper concludes that while computational decontamination is essential for high-quality scRNA-seq analysis, it must be applied judiciously. The choice of tool should be driven by the data availability (raw vs. filtered), computational resources, and prior expectations of contamination levels. The authors have made their benchmarking pipeline open-source to facilitate continuous evaluation as new tools emerge.