Assessing the impact of parental linear gene normalization on the performance of statistical models for circular RNA differential expression analysis

This study demonstrates that combining automated filtering strategies, such as edgeR's filterByExpr(), with linear-aware normalization significantly enhances the sensitivity and reproducibility of circular RNA differential expression analysis, thereby providing a standardized framework for more reliable biomarker discovery.

The Gist

Imagine your body is a bustling city, and inside every cell, there's a massive library of instructions (DNA) that tells the city how to run. Usually, these instructions are printed on long, straight strips of paper (linear RNA) that get read and then quickly recycled.

But sometimes, the library makes a mistake, or perhaps a clever trick, and glues the ends of a strip of paper together to form a perfect, unbreakable circle. These are called Circular RNAs (circRNAs).

Why do we care?

These circular papers are incredibly tough. While the straight strips rot away quickly in the body's "weather" (enzymes), the circles survive for a long time. They float around in our blood, especially in tiny blood cells called platelets. Because they are so stable and abundant, scientists think they could be the perfect "smoke signals" to detect cancer early, long before a tumor is visible on a scan.

The Problem: The Library is Messy

The problem is that finding these specific circles in a blood sample is like trying to find a few specific, rare coins in a giant pile of sand.

1. The Noise: Most of the time, the "coins" (circRNAs) are so rare that the machines counting them just see "zero."
2. The Confusion: We have different tools (software) to count these circles. Some tools are very strict and throw away anything that looks suspicious. Others are too lenient and count dust as a coin.
3. The Missing Context: Usually, we only look at the circle. But the circle is made from a straight strip of paper first. If we ignore the straight strip (linear RNA), we might miss important clues about what the circle is actually doing.

What This Study Did

The researchers in this paper acted like quality control inspectors for the tools used to find these cancer signals. They wanted to answer two big questions:

1. How strict should we be when cleaning the data? (Should we throw away weak signals, or keep everything?)
2. Does looking at the "parent" straight paper help us find the circle better?

They tested five different real-world datasets (including blood samples from breast cancer patients) and created thousands of "fake" datasets where they knew exactly which circles were supposed to be there. This allowed them to see which tools got the right answers and which ones got confused.

The Key Findings (The "Aha!" Moments)

1. The "Auto-Filter" is the Best Janitor

Imagine you are cleaning a room.

• Too Lenient (Min 1): You sweep the floor but leave every speck of dust. The room looks clean, but it's actually full of junk that confuses your eyes. The study found that being too lenient made the cancer-detection tools very unreliable. They started seeing cancer where there was none.
• Too Strict (Min 5): You throw away everything that isn't a gold bar. You might accidentally throw away a real diamond because it looked a bit small.
• The Winner (Auto-Filter): The study found that using a smart, automated cleaning tool (called filterByExpr in the software) was the sweet spot. It automatically knew how much dust to sweep away without throwing away the real treasures. This made the results much more accurate and consistent.

2. The "Parent" Connection Matters

Think of a circRNA as a child and its linear RNA as the parent.

• Old Way: We only looked at the child. If the child was quiet, we thought nothing was happening.
• New Way: The study tested a method (CIRI-DE) that looked at the child and the parent together.
• The Result: By understanding the parent's activity, the tools could hear the child's whisper much better. This approach found many more potential cancer signals than the old way of looking at the circle alone. It's like realizing that even if the child is quiet, the fact that the parent is shouting gives you a clue about what's happening in the house.

3. Platelets are the Superheroes

The researchers used blood platelets for their experiments. Platelets are like sponges that soak up these circular RNAs. Because platelets are naturally packed with these circles, they are a much better source for finding cancer signals than other blood cells. It's like trying to find a needle in a haystack (other cells) vs. finding a needle in a pile of needles (platelets).

The Bottom Line

This paper is a guidebook for scientists. It says:

• Don't be lazy with your data cleaning: Use smart, automated filters to remove the "noise" (zero counts), or your cancer detection will be full of false alarms.
• Look at the whole picture: Don't just look at the circular RNA in isolation. Use the information from the linear RNA (the parent) to help you find the signal.
• Use the right tools: Some software (like limma-voom and edgeR) handled the messy data much better than others.

By following these rules, doctors can build better "radar systems" to detect cancer early, using a simple blood test that is stable, reliable, and highly accurate. It's a step closer to catching the bad guys before they can build their fortress.

Technical Summary

1. Problem Statement

Circular RNAs (circRNAs) are promising non-invasive biomarkers for cancer due to their stability and abundance in body fluids (e.g., platelets). However, their differential expression analysis (DEA) faces significant statistical challenges:

• Zero-Inflation: circRNA data is characterized by a high proportion of zero counts and sparse Back-Splice Junction (BSJ) reads, which often deviate from the distributions assumed by standard bulk RNA-seq models.
• Lack of Standardization: There is no consensus on optimal preprocessing strategies, particularly regarding filtering thresholds (how to handle low-count/zero circRNAs) and normalization.
• Isolation of Signals: Most existing pipelines analyze circRNAs in isolation, ignoring the abundance of their parental linear RNAs. Recent tools (e.g., CIRI-DE) suggest that incorporating linear transcript information could improve normalization and detection, but this approach lacks comprehensive benchmarking.
• Performance Gaps: Standard tools (DESeq2, edgeR, limma-voom) developed for linear RNA may yield suboptimal results when applied to the unique distribution of circRNA data without parameter tuning.

2. Methodology

The authors conducted a comprehensive benchmarking study using a hybrid approach of simulated and experimental datasets.

• Datasets:
  • Experimental: Five datasets were analyzed, including three public datasets (Breast Cancer tissue, Hepatocellular Carcinoma tissue, and HCC-PBMC) and two in-house platelet RNA-seq datasets from early-stage breast cancer patients and healthy controls.
  • Simulation: Semi-parametric in silico datasets were generated using SPsimSeq to preserve the mean-variance relationship and zero-inflation of real circRNA data. These included "Zero Sets" (no DE) for Type I error testing and "DE-Signal-10%" sets (10% DE) for sensitivity testing.
• Detection & Preprocessing:
  • Two detection algorithms were used: CIRI3 and CircExplorer2 (CE2).
  • Three filtering strategies were compared:
    1. Auto-filtering: edgeR::filterByExpr() (data-driven, automated).
    2. Min 5: Minimum count of 5 reads.
    3. Min 1: Minimum count of 1 read (lenient).
• DEA Tools Evaluated:
  • Standard Bulk Models: DESeq2 (Wald, LRT, BetaPrior), edgeR (glmQLFit, glmLRT with TMM/TMMwsp), and limma-voom (various configurations).
  • Linear-Aware Models: CIRI-DE, which integrates linear RNA (FSJ) or total linear RNA counts to normalize circRNA (BSJ) expression.
• Evaluation Metrics:
  • Type I Error Control: False Positive Rate (FPR).
  • Sensitivity/Specificity: True Positive Rate (TPR), False Discovery Rate (FDR), F1 Score, and Area Under the Precision-Recall Curve (AUPRC).
  • Reproducibility: Jaccard similarity index across replicates.
  • Efficiency: Wall-clock runtime.

3. Key Contributions

• Systematic Filtering Benchmark: The study provides the first rigorous comparison of filtering strategies specifically for circRNA DEA, demonstrating that automated filtering (filterByExpr) significantly outperforms arbitrary thresholds (Min 1/Min 5).
• Validation of Linear-Aware Normalization: It empirically validates that incorporating parental linear RNA information (via CIRI-DE or modified bulk pipelines) increases the number of detected DE circRNAs compared to BSJ-only approaches.
• Tool Performance Hierarchy: The study ranks DEA tools based on their robustness to zero-inflation and filtering strategies, identifying limma-voom as the most stable performer across diverse biological contexts.
• Platelet Utility: It reinforces the utility of platelet-derived RNA as a naturally enriched source of circRNAs for liquid biopsy applications, showing higher BSJ read support compared to tissue or PBMCs.

4. Key Results

A. Impact of Filtering

• Zero Counts: Pre-filtering datasets contained 63–87% zero counts.
• Performance: Auto-filtering (filterByExpr) was superior. It reduced zero counts to <10% (often <5%), increased median BSJ read support, and significantly improved inter-method concordance (Jaccard index rose from ~0.3 to >0.8).
• Lenient Filtering: "Min 1" filtering retained excessive noise, leading to:
  • Drastic drops in sensitivity (TPR) for DESeq2 and edgeR.
  • Inflated False Discovery Rates (FDR).
  • Poor reproducibility across replicates.
• Tool Sensitivity: DESeq2 and edgeR were highly sensitive to lenient filtering, whereas limma-voom maintained stable performance even under less stringent conditions, though auto-filtering remained optimal for all.

B. Linear vs. Circular Normalization

• Detection Power: Methods incorporating linear information (CIRI-DE, FSJ+BSJ strategies) identified significantly more DE circRNAs than BSJ-only approaches. In some datasets (e.g., HCC-Tissue), BSJ-only methods found near-zero DE circRNAs, while linear-aware methods found >70.
• Concordance: While the number of DE circRNAs varied, the directionality (up/down-regulation) of the overlapping DE circRNAs was highly consistent between linear-aware and standard methods.
• Overlap: There was substantial overlap between CIRI-DE and modified bulk pipelines, suggesting that linear normalization effectively bridges the gap between specialized and standard tools.

C. Tool Ranking & Efficiency

• Best Performers: limma-voom configurations (specifically LV-Def and LV-DefRBT) demonstrated the highest stability, achieving the best balance of sensitivity, precision (F1 score), and AUPRC across all datasets and filtering strategies.
• EdgeR: Performed well with TMMwsp normalization but showed higher variability in liquid biopsy datasets (HCC-PBMC).
• DESeq2: Generally conservative; while it controlled Type I error well, it often missed true positives (lower TPR) in sparse datasets, particularly under lenient filtering.
• Runtime: edgeR was the fastest (median 0.01s per dataset), followed by limma (1.4s), while DESeq2 was the slowest (~1.37s median, up to 72 mins total for large batches).

D. Dataset Specifics

• Platelets (EBC1/EBC2): Showed the highest median BSJ read counts (20–25 reads) and best overall model performance, confirming their value as a circRNA-rich source.
• HCC-PBMC: Showed the poorest performance across all methods, likely due to low circRNA abundance and high biological heterogeneity, highlighting the challenges of liquid biopsy in non-enriched samples.

5. Significance and Conclusion

This study establishes a standardized framework for circRNA differential expression analysis, addressing critical gaps in the field:

1. Preprocessing is Critical: The choice of filtering strategy is as important as the statistical model. Automated filtering (filterByExpr) is recommended over arbitrary thresholds to remove noise and improve statistical power.
2. Linear Information Matters: Ignoring parental linear RNA leads to under-detection of DE circRNAs. Integrating linear transcript data (via CIRI-DE or custom normalization) enhances sensitivity and reliability.
3. Tool Selection: limma-voom is currently the most robust tool for circRNA DEA, offering the best trade-off between sensitivity and stability across heterogeneous datasets.
4. Clinical Implication: The findings support the use of platelet-derived circRNAs for non-invasive cancer diagnostics, provided that rigorous, linear-aware statistical pipelines are employed to distinguish biological signals from technical noise.

The authors conclude that while current bulk RNA-seq models can be adapted for circRNA analysis, the field requires continued development of specialized models that natively account for sparsity and linear-circular regulatory dynamics.