Reading Mechanism off Biological Foundation Models: An Empirical Analysis of Genomic Neighborhood Structure in scGPT Gene Embeddings
This empirical study demonstrates that the scGPT biological foundation model exhibits a small but reproducible association between its static gene embedding geometry and linear genomic neighborhood, revealing that genes located close together on the same chromosome are more similar in the model's representation than distant genes, even though this spatial information was not explicitly provided during training.