In Search of Manifolds Inside Protein Language Models: A Largely Negative Report
This paper systematically investigates the manifold hypothesis in protein language models (pLMs) using a comprehensive battery of geometric and topological diagnostics, concluding that unlike single-cell foundation models, pLM representations are predominantly high-dimensional and linear rather than organized around low-dimensional manifolds.