AI Scientists for Building Virtual-Cell Models

Technical Summary: AI Scientists for Building Virtual-Cell Models (CellForge)

Problem Statement
Predicting cellular responses to genetic and chemical perturbations is a central challenge in computational biology, essential for decoding disease mechanisms and therapeutic discovery. While the field possesses a rich toolkit of models (e.g., conditional variational autoencoders, optimal transport, graph networks), no single architecture reliably transfers across diverse perturbation types, readout modalities (transcriptomic, epigenomic, proteomic), or evaluation protocols. Current workflows require a team of human experts spanning cell biology, statistics, and deep learning to manually design bespoke predictors for each new dataset. The bottleneck has shifted from data availability to the sustained effort required for model design. The core challenge addressed is whether an autonomous system can not only automate isolated steps but complete the full hypothesis-to-validation cycle: reading literature, formulating a dataset-tailored model hypothesis, implementing it, and evaluating it against fixed, community-defined benchmarks without human design intervention.

Methodology: CellForge
The authors introduce CellForge, an organized team of autonomous agents designed for single-cell perturbation modeling. CellForge operates as a coordinated set of Large Language Model (LLM)-driven agents, each assigned specific scientific or engineering roles (data modeling, single-cell biology, deep learning, and a central critic). The system functions through three sequential stages:

1. Formulation: The system parses the input single-cell perturbation dataset (scRNA-seq, scATAC-seq, or CITE-seq), constructs held-out perturbation splits based on community-defined criteria, and reproduces established baselines. This stage locks the metric pipeline and generates a task report, ensuring the evaluation protocol is fixed before any model design begins.
2. Ideation: Expert agents retrieve relevant literature from a seed corpus and dynamic searches. They propose candidate model architectures, critique one another's assumptions, and revise the research plan through a structured, conference-review-like process. A central critic acts as an "area chair," synthesizing peer reviews into a meta-review to guide revisions. This multi-agent deliberation continues until a consensus research plan is reached, specifying the preprocessing strategy, neural architecture, loss function, and training protocol.
3. Execution: Coding agents (using the OpenHands framework) translate the approved plan into runnable Python code. The system iteratively executes the code, inspects runtime feedback, repairs errors (e.g., tensor shape mismatches, type errors), and retrains the model. The process includes 5-fold cross-validation, early stopping, and validation against the pre-defined metrics.

Crucially, the model architecture itself is treated as the scientific hypothesis. CellForge generates the architecture, loss, and training procedure end-to-end. Human involvement is strictly limited to two safety checkpoints: verifying the scientific plausibility of the research plan before execution and ensuring the training script complies with resource policies before GPU jobs are launched.

Key Contributions

• Closed Design-to-Implementation Loop: CellForge is presented as the first system to close the loop from scientific hypothesis formulation to the generation of a trained, benchmarked model in single-cell biology without manual architectural design.
• Task-Specific Architecture Generation: The system does not converge on a single universal template. Instead, it generates architectures that adapt to specific biological contexts (e.g., using diffusion models for sparse scATAC-seq data, graph attention for paired RNA-protein CITE-seq, and trajectory-aware VAEs for cytokine time-series).
• Rigorous Evaluation Protocol: Unlike systems that self-review or optimize on arbitrary splits, CellForge is evaluated against held-out perturbation conditions defined by the original dataset authors and community standards, ensuring the generated models are judged against external, non-negotiable benchmarks.

Results
CellForge was evaluated on six single-cell perturbation datasets spanning genetic (CRISPRi, CRISPRa), chemical (drug), and cytokine perturbations across scRNA-seq, scATAC-seq, and CITE-seq modalities.

• Performance on Established Benchmarks: On standard scRNA-seq benchmarks (Adamson CRISPRi, Norman CRISPRa, Srivatsan drug response), CellForge-generated models achieved performance competitive with state-of-the-art human-designed reference methods. For instance, it achieved the highest Pearson correlation (PCC = 0.9883) on the Adamson CRISPRi benchmark and the lowest differential-expression error (MSE_DE = 0.1736) on the Norman CRISPRa benchmark.
• Performance on Under-Studied Settings: In settings lacking published task-specific predictors (cytokine-driven differentiation, paired RNA-protein, and chromatin accessibility), CellForge assembled modality-aware architectures that generally outperformed generic neural baselines (Transformers, GNNs) and simple learners (linear regression, random forests).
• Plan Quality Evaluation: In blinded evaluations, both LLM judges and a panel of single-cell biology experts rated CellForge's research plans as more scientifically sound, innovative, and plausible than those generated by single LLMs or deep-research systems (e.g., OpenAI Deep Research, Perplexity).

Significance and Claims
The paper claims that CellForge demonstrates that model design itself can be treated as a scientific hypothesis. By automating the reasoning over the model space, the system produces computational methods validated on the same held-out splits as human-designed methods. The authors position this as a significant step toward autonomous scientific discovery, showing that AI agents can effectively "build another AI" in a complex biological setting that traditionally requires a multidisciplinary human team.

The authors maintain a modest tone regarding limitations:

• The system currently relies on computational validation against existing experimental data and has not yet been coupled to a wet-lab loop for prospective experimental validation.
• The number of runs per dataset (three) is limited, preventing a full characterization of the distribution of architectures the system might generate.
• Human review is minimal, and the authors acknowledge the need to test whether this minimal footprint generalizes to more complex task specifications.
• The system does not yet address spatially resolved perturbation transcriptomics, which is identified as a natural next testbed.

Ultimately, the paper argues that autonomous agents can complete the full hypothesis-to-validation cycle in single-cell biology, offering a new paradigm for method design in domains where no dominant modeling paradigm yet exists.