Technical Summary: Chem-PerturBridge

Problem Statement

Large-scale perturbation models require training data that encompasses diverse chemical, cellular, and assay conditions to effectively predict cellular responses, generalize across contexts, and learn reusable representations. However, existing transcriptomic resources for small-molecule modeling are fragmented. Public and semi-public datasets differ significantly in profiling technologies (e.g., bulk RNA-seq, LINCS L1000, DRUG-seq, single-cell), measured genes, control designs, dose ranges, timepoints, replicate structures, metadata conventions, and preprocessing pipelines.

This fragmentation creates two primary challenges:

1. Diagnostic: It is unclear whether transcriptomic effects measured for the same compound in the same cellular context agree across different datasets after harmonization.
2. Constructive: It is uncertain whether training models on multi-dataset, heterogeneous data can extract signals that improve downstream predictions across datasets, or if the noise introduced by assay differences outweighs the benefits of scale.

Methodology

The authors introduce Chem-PerturBridge, a harmonized multi-dataset resource designed to facilitate the training and evaluation of small-molecule transcriptomic perturbation models.

Resource Construction

Chem-PerturBridge integrates over 37,000 compounds, 136 cellular contexts, and 1.25 million transcriptomic samples across eight assay types, including bulk RNA-seq, LINCS L1000, DRUG-seq, targeted count assays, and pseudobulked single-cell data. The harmonization pipeline follows four design principles:

• Explicit Identifiers: Compounds are mapped to PubChem CIDs; cellular contexts to Cellosaurus or Cell Ontology IDs; genes to Ensembl 115 identifiers; and doses/timepoints are standardized to micromolar and hours, respectively.
• Replicate-Aware Effects: The resource generates two sets of signatures: one aggregating all treated and control replicates, and another preserving individual replicate profiles to study consistency.
• Assay-Specific Processing:
  • Count-based data (e.g., DRUG-seq, single-cell pseudobulks) undergoes quality control, TMM normalization, voom transformation, and differential expression analysis using limma.
  • Normalized L1000 profiles (Level 3) are processed directly using limma without count-based normalization steps.
• Calibrated Evaluation: Agreement metrics are interpreted relative to "same-context, different-compound" baselines to account for shared biological or technical structures.

Evaluation Framework

The paper employs two complementary evaluation strategies:

1. Signature Agreement:
   • Cross-Dataset: Measures agreement between the same compound in the same context across different datasets.
   • Within-Dataset: Measures agreement between technical/biological replicates of the same condition.
   • Metrics: Spearman correlation of log fold-changes (logFC), direction agreement (sign of logFC), and retrieval scores (identifying the true same-compound partner among distractors).
2. Representation Learning:
   • OP3 Benchmark: Pretrained compound embeddings from Chem-PerturBridge are evaluated on an unseen-compound task in the OP3 dataset. Models are compared against L1000-only embeddings, Morgan fingerprints, and descriptor-free baselines.
   • Molecule-Holdout LPM: A multi-output Large Perturbation Model (LPM) is trained on Chem-PerturBridge and evaluated on held-out molecules across 11 source datasets to test direct transferability.

Key Results

1. Signature Agreement is Weak in Fine-Grained Metrics

• LogFC Magnitude: Fine-grained logFC rankings and magnitudes show weak agreement across most dataset pairs. In many cases, the agreement between the same compound in different datasets falls below the baseline agreement between different compounds in the same context.
• Direction Agreement: In contrast, the agreement in the direction of logFC (upregulation vs. downregulation) is substantially more stable and typically exceeds the different-compound baselines.
• Retrieval: While cross-dataset retrieval of same-compound partners is better than random, the same-context different-compound baseline often ranks higher than the true match, indicating that context-level and assay-level structures are strong confounders.
• Replicate Consistency: Within-dataset replicate agreement for DEG-restricted logFC Spearman correlation is often negative after baseline adjustment, suggesting that technical replicates are sometimes less similar than different compounds in the same context. However, direction agreement remains high and stable across replicates.

2. Representation Learning Benefits from Harmonized Data

Despite weak direct signature agreement, pooled training on Chem-PerturBridge yields transferable compound representations:

• OP3 Evaluation: Embeddings pretrained on Chem-PerturBridge outperform L1000-only embeddings, Morgan fingerprints, and descriptor-free baselines across Spearman correlation, Cosine similarity, MRRMSE, and MAE metrics on held-out OP3 compounds.
• Molecule-Holdout Evaluation: Models trained on the full Chem-PerturBridge corpus, particularly when initialized with Morgan fingerprints and fine-tuned on target datasets with frozen molecule embeddings, achieve the lowest or tied-lowest test RMSE in 10 out of 11 evaluated datasets. This suggests that heterogeneous data can recover transferable compound-level signals even when direct signature matching is poor.

Significance and Claims

The paper claims that Chem-PerturBridge serves a dual purpose:

1. Diagnostic Benchmark: It provides a rigorous framework to demonstrate that raw perturbation signatures are not directly interchangeable across assay families. Fine-grained logFC agreement is weak, whereas direction agreement is more robust. This distinction is critical for interpreting cross-dataset comparisons and designing evaluation metrics.
2. Model-Ready Substrate: It validates that a harmonized compendium of heterogeneous data can be used to pretrain models that learn reusable compound representations. The study demonstrates that "weak direct logFC agreement does not rule out representation learning from heterogeneous resources."

The authors conclude that while assay, batch, and processing effects dominate fine-grained signature agreement, compound-level signals can still be recovered by models trained across diverse, well-annotated data, especially when combined with molecular priors and target-domain adaptation. The resource supports both the diagnostic evaluation of cross-dataset agreement and the development of foundation models for transcriptomic perturbation.