Rectangle: robust and scalable multiscale deconvolution informed by single-cell RNA sequencing data

Technical Summary: Rectangle – Robust and Scalable Multiscale Deconvolution Informed by Single-Cell RNA Sequencing Data

Problem Statement
Bulk RNA sequencing (RNA-seq) enables large-scale profiling of complex tissues but lacks cellular resolution. While in silico deconvolution methods infer cell-type composition from bulk data, current second-generation methods (which utilize single-cell RNA-seq [scRNA-seq] references) face three critical limitations:

1. Resolution: They often fail to distinguish closely related cell phenotypes or states due to transcriptional similarity.
2. Scalability: They do not scale efficiently to the rapidly expanding volumes of large scRNA-seq datasets (e.g., >100k cells), often suffering from high computational costs or memory constraints.
3. Robustness to Unknowns: They typically assume the reference signature matrix covers all cellular content in the bulk sample. When "unknown cellular content" (cell types or states absent from the reference) is present, these methods often misattribute signals, leading to biased fraction estimates.

Methodology
The authors present Rectangle, an open-source Python framework within the scverse ecosystem designed for recursive disenTANGLEment of bulk RNA-seq data. The algorithm operates in two primary phases:

1. Signature Matrix Construction (Bootstrapped Pseudobulking):

   • Pseudobulking: Instead of processing every single cell, Rectangle bootstraps the scRNA-seq reference by repeatedly sampling a small, constant number of cells (default 500) with replacement for each annotated cell type to create "pure" pseudobulk expression profiles. This ensures a small memory footprint and runtime independent of the total reference size.
   • Marker Selection: Differential expression (DE) analysis is performed using pyDESeq2. Log-fold-change (LFC) thresholds are adaptively optimized via grid search on synthetic pseudobulk mixtures to maximize correlation with ground truth.
   • Optimization: The resulting signature matrix is refined by removing poorly discriminative genes to minimize multicollinearity (condition number) and scaled by cell-type-specific mRNA content to prevent systematic bias.
   • Multiscale Strategy: Two signature matrices are generated: a "direct signature" for fine-grained cell types and a "clustered signature" where transcriptionally similar types are grouped via hierarchical clustering.

2. Deconvolution (Hierarchical Quadratic Programming):

   • Unknown Content Estimation: The method explicitly models unknown cellular content as the residual between observed bulk expression and the reconstruction based on the signature matrix.
   • Hierarchical Deconvolution: For highly similar cell types, a two-step process is employed:
     • Step 1: A quadratic programming (QP) solver deconvolves the bulk data using the "clustered signature" to estimate coarse cell-group proportions.
     • Step 2: A second QP step uses the "direct signature" to resolve fine-grained fractions, constrained by the cluster-level estimates from Step 1 within a narrow tolerance interval. This restricts the solution space for similar types while enabling high resolution.
   • Final Scaling: The final known cell fractions are rescaled such that they, combined with the estimated unknown component, sum to 100%.

Key Contributions

• Multiscale Framework: Rectangle uniquely combines hierarchical deconvolution with explicit modeling of unknown cellular content, addressing both resolution and robustness simultaneously.
• Scalability via Bootstrapping: By decoupling runtime from the total number of single cells in the reference (relying instead on the number of cell types and genes), Rectangle achieves linear memory scaling and near-constant runtime scaling relative to reference size.
• Integration: As part of the scverse ecosystem, it supports AnnData and SpatialData formats, ensuring interoperability with standard single-cell and spatial transcriptomics tools.

Results
The authors evaluated Rectangle against eight state-of-the-art methods (including CIBERSORTx, Scaden, MuSiC, BayesPrism) using the omnideconv benchmarking framework across diverse datasets (mouse/human tissues, tumor microenvironments, spatial transcriptomics).

• Accuracy and Resolution: Rectangle achieved the highest overall Pearson correlation (mean 0.804) and lowest cumulative Root Mean Square Error (RMSE) across all validation datasets. It demonstrated low spillover (15.81% average), comparable to top performers, and successfully resolved fine-grained phenotypes in complex tumor ecosystems (e.g., distinguishing myofibroblastic vs. inflammatory CAFs) where coarse methods failed.
• Computational Efficiency: On a standard laptop (Apple M1, 32GB RAM) with a ~150k-cell reference, Rectangle completed the full pipeline (signature building + deconvolution) in approximately one minute. In contrast, other methods failed to execute or required significantly longer runtimes and memory as reference sizes increased. Rectangle was the only method to maintain low runtime and memory usage across a 150-fold increase in reference size.
• Unknown Content Handling: In simulations with increasing unknown content, Rectangle was the only method maintaining high global correlation and low RMSE for reference-covered types. In real-world applications:
  • Lung Cancer: When tumor cells were excluded from the reference, the estimated "unknown" fraction correlated strongly with consensus tumor purity (r=0.612). Combining this with reference-guided tumor estimates improved purity quantification (r=0.772).
  • Brain Organoids: Applied to time-series bulk data against a mature-cell reference, Rectangle identified high "unknown" fractions at early time points. Reconstruction error analysis of these unknowns revealed coherent early developmental programs (e.g., neurogenic differentiation, cell-cycle regulation), effectively disentangling hidden progenitor states.
• Spatial Transcriptomics: Without specific spatial adaptations, Rectangle successfully deconvolved 10x Visium mouse brain data, accurately mapping fine-grained neuronal subtypes to their anatomical regions and ranking among top performers in the Spotless benchmark.

Significance
The paper claims that Rectangle bridges the gap between the high resolution of single-cell transcriptomics and the scale/cost-efficiency of bulk RNA-seq. By simultaneously addressing accuracy, scalability, and reference incompleteness, it enables population-scale cellular profiling. This capability supports the discovery of cellular biomarkers and the tracking of cell-state dynamics in settings (such as large clinical cohorts or longitudinal studies) where comprehensive single-cell sequencing is impractical. The authors position Rectangle as a foundational tool for moving beyond reference-matched quantification to recover "hidden" biological signals, such as malignant heterogeneity and developmental trajectories.