SVC-Probe: A Framework for Evaluating Perturbation Generalization in Spatial Foundation-Model Embeddings

Technical Summary: SVC-Probe: A Framework for Evaluating Perturbation Generalization in Spatial Foundation-Model Embeddings

Problem Statement
While spatial foundation models have advanced the acquisition of concise representations for protein organization from large-scale microscopy data, their evaluation remains limited. Current benchmarks predominantly assess baseline localization, retrieval, or the ability to discriminate between observed experimental conditions. However, a critical gap exists in evaluating whether these learned representations capture mechanistic structures that persist under intervention. Specifically, it is unclear if spatial embeddings can predict out-of-sample perturbation responses or recover drug-induced reorganization patterns that transfer across different chemical agents. In spatial proteomics, this requirement is stringent, as treatment effects involve not only protein displacement but also coordinated subcellular remodeling and neighborhood reorganization.

Methodology: The SVC-Probe Framework
The authors introduce SVC-Probe, a diagnostic framework designed to assess perturbation generalization in spatial foundation-model embeddings. The framework integrates three complementary modules applied to the CM4AI MDA-MB-468 chemical-perturbation atlas (462 antibody labels, control, paclitaxel, and vorinostat conditions) using 1536-dimensional SubCell embeddings:

1. Subcellular Embedding Atlas Stability (SEAS): This module quantifies the stability of individual protein embeddings under perturbation. It computes a stability score $S(P) \in (0, 1]$ based on geodesic centroid drift and the spread ratio of per-cell embedding clouds. Proteins are categorized as stable, plastic, or mid-tier based on bootstrap confidence intervals.
2. Mondrian Neighborhood Graphs (MNG): This module analyzes the reorganization of local co-localization neighborhoods. It constructs a label-by-label cosine-similarity matrix to identify mutual top-$k$ neighbors and calculates a turnover metric $R(P)$ representing the fraction of neighbors lost or gained under treatment. Leiden community detection is used to delineate functional subsystems, visualized as Mondrian-style layouts.
3. Foundation Model Perturbation Probe (FMP): This module tests the predictive capability of the latent space. A regression head is trained to predict the drug-condition centroid $\hat{\mu}(P,c)$ using the control centroid $\mu(P,ctrl)$ and a learned perturbation token. The framework employs a Leave-One-Drug-Out (LODO) evaluation strategy to assess cross-drug transferability, distinguishing between in-domain fitting and genuine generalization. It also utilizes a "null calibration" approach, comparing real drug coupling against control-vs-control splits to isolate drug-specific signals from generic embedding geometry.

Key Results

• Classification vs. Generalization: While SubCell embeddings achieved high accuracy (98.6%) in distinguishing the three observed drug conditions, this performance did not translate to perturbation generalization. In the LODO setting, the learned perturbation probe's cosine similarity dropped drastically from 0.944 (in-domain) to 0.30, indicating a failure to transfer learned perturbation structures to unseen drugs. In contrast, simple baselines (identity and global shift) remained stable, suggesting the learned model captures drug-specific artifacts rather than transferable mechanisms.
• Null Calibration: Raw correlation between FMP residuals and neighborhood turnover was found to be driven largely by generic embedding structure ($\rho \approx 0.21$ in null splits). After null calibration, a drug-specific signal emerged for vorinostat ($\rho = -0.186$), consistent with chromatin-related reorganization, whereas no consistent pattern was observed for paclitaxel, likely due to sparse coverage of microtubule-associated proteins in the panel.
• Stability and Rewiring: On a 33-label calibration subset, SEAS correctly identified chromatin modifiers as "plastic" under vorinostat while structural controls remained stable. Furthermore, partial Spearman correlation analysis confirmed that neighborhood turnover provides information about the rewiring axis distinct from raw centroid drift.

Significance and Claims
The paper claims that SVC-Probe serves as a reusable diagnostic workflow for stress-testing spatial virtual-cell representations. The primary significance lies in demonstrating that strong condition discrimination does not imply perturbation generalization. The authors argue that perturbation generalization should serve as a stricter and more informative benchmark than baseline condition discrimination.

The framework highlights that while current spatial foundation models can effectively separate observed states, they may fail to preserve biologically informative structure under intervention. The results suggest that perturbation-aware analysis is necessary to validate whether embeddings capture mechanistic axes (such as the chromatin axis under HDAC inhibition) rather than generic geometric artifacts. The authors position these findings as hypothesis-generating, noting that the observed signals require further validation across broader drug classes and cell lines.