Technical Summary: Plausibility Is Not Prediction: Contrastive Evidence for LLM-Based Cellular Perturbation Reasoning

Problem Definition and Motivation

Perturbation experiments, such as drug treatments and genetic editing, are fundamental to understanding cellular mechanisms but remain costly and sparse. This limitation motivates the prediction of gene expression responses for unobserved conditions. While recent work has proposed using Large Language Models (LLMs) as "virtual cell" simulators to perform stepwise, knowledge-grounded mechanistic reasoning for differential expression (DE) prediction, this paper identifies a critical failure mode: plausibility is not prediction.

Existing LLM-based methods (e.g., VCWorld, SUMMER) typically evaluate perturbation–gene pairs in isolation. They prompt models to generate biologically plausible rationales for whether a gene is differentially expressed given a perturbation. However, the authors demonstrate that these methods systematically fail to capture perturbation-specific effects. Instead of reasoning about the specific intervention, models rely on intrinsic gene response tendencies (gene-frequency priors). This leads to three empirical failure modes:

1. Systematic Overestimation: Models exhibit a strong bias toward predicting "yes" (differential expression), often predicting positive outcomes for the majority of queries even when the empirical positive rate is low (e.g., predicting "yes" for 92.1% of queries when the true rate is 29.0%).
2. Confounded Aggregate Metrics: High aggregate performance (AUROC/AUPRC) is often achieved by exploiting the marginal frequency of gene responses (some genes are rarely expressed, others frequently) rather than learning to distinguish between different perturbations for the same gene.
3. Chance-Level Per-Gene Discrimination: When evaluated on a macro-per-gene basis (requiring the model to rank different perturbations for a fixed gene), isolated LLM reasoning collapses to chance-level performance, indicating a lack of true perturbation-specific reasoning.

The core issue identified is the organization of evidence. Existing methods present queries in isolation without exposing how related perturbations differ in their effects on the same target gene. Without contrastive signals, models cannot distinguish perturbation-specific outcomes from general biological plausibility.

Methodology: CORE

To address these limitations, the authors propose CORE (Contrastive Organization of Relational Evidence), a framework that reframes perturbation prediction as a comparison task.

1. ReasonKG: A Harmonized Biomedical Knowledge Graph

CORE relies on ReasonKG, a newly constructed knowledge graph that harmonizes five biomedical resources (VCWorld, PrimeKG, HetioNet, PertKGE, and BioKG). ReasonKG integrates 667,920 canonical nodes and over 27 million directed edges, creating a dense, transferable graph space that connects drugs, genes, pathways, and diseases. This integration resolves connectivity gaps found in individual graphs (e.g., VCWorld's isolated drug-gene pairs), enabling robust retrieval of biologically related perturbations.

2. Contrastive Evidence Construction

For a held-out query $(p, g)$ (perturbation $p$, gene $g$), CORE performs the following steps:

• Retrieval: It retrieves support perturbations $p'$ that are biologically related to $p$ using KG-derived signatures (based on shortest-path distances to grounded genes).
• Contrastive Organization: It collects the observed DE labels for these related perturbations on the same target gene $g$.
• Partitioning: The retrieved evidence is partitioned into:
  • Positive Support Set ($H^+$): Related perturbations that caused differential expression in $g$.
  • Negative Support Set ($H^-$): Related perturbations that did not cause differential expression in $g$.
• Scoring: The evidence is summarized into vote masses ($V^+, V^-$) based on similarity weights, generating a contrast score ($s_{contrast}$) that reflects the proportion of positive evidence.

3. Framework Instantiations

CORE is instantiated in two complementary settings to validate the utility of contrastive evidence:

• CORE-Voting (Non-LLM): A deterministic aggregator that computes a weighted combination of the same-gene contrast score and a gene-anchor smoothed score (borrowing evidence from genes with similar response profiles). This tests whether the contrastive signal itself contains perturbation-specific information independent of LLM reasoning.
• CORE-Reasoning (LLM-based): This reformulates the structured evidence packet (including contrast scores, positive/negative support sets, KG paths, and gene-anchor scores) into a prompt for an LLM. The LLM is tasked with adjudicating the quality of the evidence (e.g., distinguishing specific mechanistic paths from generic connectivity) and calibrating the final prediction, rather than generating unconstrained plausibility.

Key Results

The authors evaluate CORE on two benchmark families: Tahoe100M (drug perturbations in C32 melanoma cells) and PerturbQA (CRISPRi gene perturbations in K562, HepG2, Jurkat, and RPE1 cells).

1. Calibration and Bias Correction

• Reduction of "Yes" Bias: Standard VCWorld-style prompts often predict "yes" for >80% of queries. CORE-Reasoning significantly reduces this bias, bringing the predicted positive rate close to the empirical prevalence (e.g., reducing Mistral-7B's "yes" rate from 92.1% to 26.1% on C32).
• Improved Calibration: CORE-Reasoning improves aggregate AUROC, accuracy, and F1 scores across various LLM backbones (Mistral, Llama, DeepSeek, Qwen3.5, GPT-5.4). For example, on C32, CORE-Reasoning with Qwen3.5-9B improves aggregate AUROC by 26.0%, accuracy by 19.8%, and F1 by 28.6% compared to standard prompting.

2. Perturbation-Specific Discrimination

• Per-Gene Performance: The most significant gain is in macro-per-gene metrics, which measure the ability to distinguish perturbations for a fixed gene.
  • CORE-Voting: Without an LLM, CORE-Voting improves macro-per-gene AUROC from 0.500 (chance) to 0.574 (fixed budget) and 0.627 (full support) on C32, and to 0.711 on PerturbQA K562.
  • CORE-Reasoning: LLMs leveraging contrastive evidence also show substantial per-gene improvements, confirming that the evidence structure enables better discrimination than isolated reasoning.

3. Comparison with Baselines

• Against Gene-Prior Voting: While simple gene-frequency baselines perform well on aggregate metrics, they fail completely on per-gene discrimination (AUROC = 0.500). CORE outperforms these baselines on per-gene metrics while maintaining competitive aggregate performance.
• Against State-of-the-Art: CORE-Reasoning outperforms recent state-of-the-art single-cell perturbation models (e.g., STATE, PerturbDiff) and prior LLM-based methods (VCWorld, SUMMER) on most metrics, particularly in the challenging out-of-distribution setting where test perturbations are held out from training.

Significance and Claims

The paper claims that contrastive evidence organization is essential for reliable LLM-based perturbation reasoning. The central finding is that biological plausibility alone is insufficient for prediction; models require empirical contrast—specifically, evidence showing how similar perturbations yield different outcomes for the same gene—to learn perturbation-specific effects.

By organizing evidence into positive and negative support sets for the same target gene, CORE shifts the paradigm from "generating a plausible story" to "grounded comparison." This approach:

1. Decouples Plausibility from Prediction: It prevents models from treating any biologically connected path as sufficient evidence for differential expression.
2. Improves Calibration: It corrects the systematic overestimation of positive outcomes common in isolated LLM prompting.
3. Enables Transfer: The framework improves performance across different cell lines and perturbation types (drugs and genetic edits), suggesting the utility of contrastive organization extends beyond specific datasets.

The authors conclude that reliable "virtual cell" reasoning depends as much on the structure of evidence as on the coverage of biological knowledge, highlighting that the bottleneck in current LLM-based approaches is not model capacity or knowledge retrieval, but the lack of contrastive signals required for specific reasoning.