pysigscore: gene signatures scoring across bulk and single-cell transcriptomics

Technical Summary of pysigscore: gene signatures scoring across bulk and single-cell transcriptomics

Problem Statement
High-throughput transcriptomics has established gene signatures as central tools for interpreting gene expression data in diagnosis, prognosis, and biological function characterization. However, quantifying signature activity and assessing its robustness remain challenging. Existing scoring methods rely on varying assumptions regarding background distributions, robustness, and statistical foundations (e.g., simple summaries, Z-statistics, or rank-based approaches). Consequently, no single scoring approach is universally optimal for all datasets or biological questions. There is a distinct need for a unified framework that allows researchers to compute, compare, and benchmark multiple scoring methods while evaluating their statistical significance and reliability.

Methodology
The authors present pysigscore, a Python framework designed for gene set scoring in both bulk and single-cell RNA-seq (scRNA-seq) data. The framework is coordinated by a Scorer() object that manages a unified workflow comprising data preprocessing, score computation, reliability assessment, and visualization.

• Input and Preprocessing: The system accepts expression matrices (as DataFrames or AnnData objects) and gene sets (as dictionaries or binary matrices). It supports extensive preprocessing, including log transformation, standardization, min-max scaling, k-Nearest-Neighbours (k-NN) smoothing, and step/quantile transformations.
• Scoring Engine: The core functionality is driven by the compute_scores() function, which generates a score matrix where rows represent samples/cells and columns represent gene sets. The framework implements 18 built-in scoring methods organized into three families:
  1. Direct Statistics (14 methods): Includes Sum, WeightedSum, Mean, Median, Mode, and robust statistics like IQR, MAD, and AAD. These compute scalar summaries directly from signature gene values.
  2. Z-score (1 method): Standardizes the mean expression of a gene set relative to the full expression profile of the sample.
  3. Rank-based Methods (3 methods): Includes AUCell, ssGSEA, and GSVA, which assess enrichment of signature genes among the highest-ranked genes within a sample.
• Customization: A key feature is the Custom Scorer, which allows users to define new scoring functions via Python code without modifying the library source. These functions must accept the signature gene expression vector and the full sample expression vector.
• Reliability and Significance:
  • P-value Estimation: For direct methods, significance is estimated via permutation testing using size-matched random gene sets to generate a null distribution, applying a plus-one correction. Z-scores use analytical two-sided p-values from the standard normal distribution. Rank-based methods utilize empirical procedures or backend implementations.
  • Leave-One-Out (LOO) Experiments: Robustness is assessed by recomputing scores after iteratively removing each gene in a signature. Deviations are quantified using Mean Absolute Error (MAE), Mean Squared Error (MSE), or Mean Absolute Percentage Error (MAPE) to identify dependence on specific "hub" genes.

Key Contributions
The paper claims five primary contributions that extend previous work (such as the R package sigscores and sigQC):

1. Integration of scRNA-seq Methods: It incorporates new approaches specifically developed for single-cell transcriptomics, expanding the built-in method count to 18.
2. Unified Customization: It provides a flexible built-in custom scorer, enabling users to implement, benchmark, and directly compare their own scoring functions against existing approaches within a single library.
3. Gene-Level Contribution Analysis: It facilitates the evaluation of individual gene contributions to signature activity through LOO experiments.
4. Scalability: The framework is optimized to handle large transcriptomic datasets efficiently.
5. Interoperability: It is directly compatible with major RNA-seq analysis frameworks (e.g., AnnData), ensuring seamless integration into existing pipelines.

Results
The authors validated pysigscore using the CCLE, TCGA, and PBMC datasets. The validation demonstrated the framework's ability to recover expected biological enrichments, specifically in liver, hypoxia, inflammatory, and cell-cycle signatures. The results confirm that the framework can successfully process diverse data types and yield consistent, interpretable scores across different experimental settings.

Significance
The paper positions pysigscore not merely as a tool for calculation, but as a reproducible framework for methodological comparison. By allowing users to compute, compare, and assess the robustness of different scoring methods simultaneously, it addresses the limitation that no single approach is optimal for all contexts. The tool empowers researchers to move beyond selecting a single "best" method a priori, instead enabling data-driven decisions on which scoring strategy best fits specific biological questions and dataset characteristics.