🧬 biology

Geometric Representations of Knowledge Inside Biological Large Language Models: an Empirical Analysis

This empirical study reveals that while biological large language models (SCFMs) do encode a real, low-dimensional, and partially linearized geometric structure for biological knowledge, this structure is modest, often nonlinearly entangled, and largely overlaps with what classical expression-based methods like PCA can already recover, falling short of the crisp, regular geometry observed in natural language models.

Original authors: Olivia Denvis

Published 2026-07-22

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Olivia Denvis

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you have a giant, magical library where every book is a single living cell from the human body. For years, scientists have tried to read these books to understand how our bodies work, but the text is messy and hard to decipher. Recently, a new kind of "super-reader" called a Biological Large Language Model (or Biological LLM) was invented. These are computer programs trained on millions of cell "books" to spot patterns, much like how your phone's keyboard learns to predict the next word you'll type.

The big question scientists have been asking is: Do these super-readers organize knowledge in a neat, straight-line way? In regular language models (the ones that write essays or chat with you), researchers found that ideas like "king" and "queen" sit on a straight line, and opposites are just a simple step away from each other. It's like a perfectly organized map where every concept has its own clear street. Scientists hoped these biological super-readers would have the same neat, straight-line map for things like "healthy vs. sick" or "growing vs. resting." If they did, it would mean we could easily tweak the computer's thinking to understand diseases or development. But, as we are about to see, the map inside these biological models is a bit more like a winding, hilly forest path than a straight city street.

The Map Inside the Machine: A Forest, Not a Highway

In this study, a researcher named Olivia Denvis decided to take a deep dive into four of these biological super-readers (named scBERT, Geneformer, scGPT, and UCE) to see if they really have that neat, straight-line organization. She treated them like explorers, sending them into a massive dataset of 420,000 cells with detailed notes on what they are, where they live, and what they are doing. She then compared the models' internal "maps" against the old-school, reliable tools scientists have used for decades, like simple math tricks called PCA.

Here is what she found: The models do have a map, but it's not the crisp, straight highway everyone hoped for.

1. The Map is Compact, but Crowded
First, the researchers checked how much space the models use to store information. Imagine a huge warehouse (the model's full size) that could hold 1,280 shelves. The study found that the models only actually use about 12 to 26 shelves to store their knowledge. That's a very low-dimensional space, which is good because it means the models are efficient. However, the space is "anisotropic," which is a fancy way of saying it's shaped like a long, narrow cone rather than a round ball. The smaller models were even more squeezed into this narrow cone, suggesting they might be a bit "cramped" in how they think.

2. The Easy Stuff is Straight, The Hard Stuff is Curvy
When the researcher asked the models to identify simple things, like "Is this cell from a male or female?" or "Is it in the immune system?", the models were amazing. They could draw a straight line to separate these groups, just like the language models do. In fact, for these easy categories, the models were almost perfect.

But here is the twist: When she asked about the interesting stuff—like "Is this cell sick?" or "What specific subtype is this?" or "How far along is it in its development?"—the straight lines stopped working. The knowledge was still there, but it was tangled up in curves.

The Proof: When the researcher tried to use a simple straight line to predict a cell's development time, it only got about 61% of the answer right. But when she let the computer follow the natural, curved path of the data (like walking along a winding trail instead of cutting through a field), the accuracy jumped to 80%.
The Takeaway: The models know the complex stuff, but they store it in a curved, non-linear way that simple straight lines can't easily reach. This is different from language models, where even complex ideas often sit on straight lines.

3. The "Steering Wheel" is a Bit Wobbly
In language models, if you want to change a word's meaning (like turning "happy" into "sad"), you can just add a specific vector (a direction) to it, and it works perfectly. The researcher tried this with the biological models. She tried to "steer" a cell's identity by adding a direction vector.

The Result: It worked, but only about 54% to 66% of the time. It was better than a coin flip, but far from the perfect control seen in language models. Sometimes, trying to change one thing accidentally changed something else. The directions for different ideas were somewhat parallel, but not perfectly so, and they were only weakly aligned.

4. The Models Agree with Each Other, Not the "Truth"
The researcher also checked if all four models were thinking the same way. They were moderately similar to each other (about 55% to 74% similar), which is nice. But when she compared them to the "Gold Standard" of biological knowledge (a detailed map of how cell types are related in real life), the models were only about 36% to 45% similar.

The Surprise: The models actually looked more like the old-school math tools (PCA) than they looked like the real biological truth. They converged on a shared, simplified view of the data that was closer to the basic math than to the complex reality of biology.

5. The "Deep" Knowledge is in the Middle
Finally, she looked at where in the model's layers (its "brain") this knowledge lived.

Simple Identity: Knowing "what kind of cell" this is gets stronger the deeper you go into the model.
Disease Status: Knowing if a cell is sick peaked in the middle layers and then faded away as the model got deeper.
Batch Effects: The models were good at ignoring technical noise (like which machine took the picture) in the early layers, but they didn't completely forget it.

The Bottom Line

So, what's the verdict? Biological Large Language Models are real, and they do hold a geometric map of knowledge. But it's not the clean, straight-line, perfectly organized map we saw in language models. Instead, it's a "partially linearized, low-dimensional" map.

The models are great at the easy stuff (like telling a male cell from a female one), but for the hard, clinically important stuff (like disease or development), the knowledge is curved and tangled. Most of the "straight-line" knowledge the models have is actually just what we could already get from simple, old-school math tools. The new stuff the models learned is there, but it's hidden in the curves, making it harder to read and use.

The study concludes that while these models are useful, they aren't the magic "straight-line" breakthrough some hoped for. The real challenge for the future isn't just building bigger models, but building better tools to read the curved, winding paths where the most important biological secrets are actually hiding.

Technical Summary: Geometric Representations of Knowledge Inside Biological Large Language Models

Problem Statement
Large Language Models (LLMs) exhibit a "linear representation hypothesis," where high-level knowledge (concepts, analogies, taxonomies) is encoded in strikingly regular geometric structures: concepts align with linear directions, analogies with parallel offsets, and hierarchies with nested, orthogonal subspaces. Single-cell foundation models (SCFMs)—biological LLMs trained on masked transcriptome sequences—have emerged as repositories of biological knowledge. However, it remains untested whether these biological models inherit the crisp, linearizable geometry observed in language models, or if their internal representations are merely correlational and entangled. Furthermore, it is unclear how much of any observed structure is genuinely novel to the foundation model versus what is already recoverable from classical linear expression embeddings (e.g., PCA).

Methodology
The authors conducted a systematic empirical analysis comparing four SCFMs of varying scale (8.9M to 650M parameters: scBERT, Geneformer, scGPT, UCE) against strong linear expression baselines (Highly Variable Genes, PCA-50, and scVI). The evaluation utilized a curated 0.42M-cell atlas panel with rich metadata (cell type, tissue, sex, disease, etc.) and a hematopoietic differentiation trajectory dataset with ground-truth pseudotime.

The study employed six families of geometric analyses, with every measurement explicitly compared against the expression baselines to isolate model-added structure:

Global Geometry: Estimation of intrinsic dimension (via TwoNN and Levina–Bickel MLE) and anisotropy (mean cosine similarity of random pairs).
Probing: Linear (logistic regression) and nonlinear (MLP) probes to measure the decodability of nine attributes, calculating the gap between linear and nonlinear performance.
Concept Directions: Analysis of analogical parallelism (cosine similarity of difference-in-means vectors across lineages), near-orthogonality of unrelated concepts, and steering success rates, evaluated in both raw and causally whitened spaces.
Categorical & Cross-Model Geometry: Assessment of hierarchical alignment (cophenetic correlation to Cell Ontology) and cross-model similarity using Centered Kernel Alignment (CKA) against both other models and an ontology reference kernel.
Trajectory-Manifold Geometry: Comparison of pseudotime recovery via a single linear direction versus a geodesic (manifold) estimate on the hematopoietic dataset.
Depth Profiling: Layer-wise analysis of where specific knowledge (cell type, disease, batch) concentrates within the network.

Key Results
The study finds that the geometry of knowledge in biological LLMs is real but modest, diverging significantly from the "crisp" structure documented in language models:

Low-Dimensional and Anisotropic: SCFM representations are low-dimensional (intrinsic dimension 12–26, or 4–6% of model width) and strongly anisotropic (mean cosine 0.29–0.52), contrasting with the near-isotropic nature of PCA/scVI baselines.
Linear Decodability is Limited: Coarse categorical knowledge (compartment, sex, broad cell type) is linearly decodable but offers negligible incremental gain over PCA (median gain of ~3.5 balanced accuracy points). The models add the most value to intermediate distinctions (subtypes) but offer no advantage for already-easy labels (sex) or batch variables (where they sometimes perform worse than PCA).
Nonlinear Entanglement: Finer attributes (disease state, fine subtypes, developmental time) are present in the representations but are nonlinearly entangled. The gap between linear and nonlinear probes reaches 9 points for disease, and a single linear direction recovers only $R^2 \approx 0.61$ of pseudotime, whereas a curved geodesic reaches $\rho \approx 0.80$ (matching classical diffusion maps).
Weak Linear Structure: Concept directions are only weakly parallel (mean cosine 0.17–0.26, rising to 0.28–0.41 after causal whitening) and steering succeeds only 54–66% of the time. However, unrelated concepts are near-orthogonal, preventing total collapse of distinct concepts.
Partial Taxonomic Alignment: Hierarchical organization is present but coarse (cophenetic correlation ~0.58), far from the clean nested simplices seen in language models.
Convergence: Cross-model similarity is moderate (CKA 0.55–0.74), but models converge more strongly to each other and to expression baselines than to the biological ground truth (Cell Ontology kernel CKA 0.36–0.45).

Significance and Claims
The paper concludes that current biological LLMs encode a partially linearized, low-dimensional geometry that is useful but far from the idealized structure of language models. The primary significance of these findings is threefold:

Explanation of Benchmark Margins: The "slim margins" often observed in SCFM benchmarks over classical pipelines are explained by the fact that the linearly accessible knowledge in these models largely overlaps with what PCA already captures. The additional, more complex knowledge (disease, development) is encoded nonlinearly, making it inaccessible to standard linear downstream heads.
Methodological Correction: The authors argue that applying LLM interpretability tools to biology without a baseline is misleading. In single-cell analysis, where linear embeddings are already highly informative, the critical question is not if structure exists, but how much the foundation model adds beyond classical methods.
Future Direction: The results suggest that the most decision-relevant knowledge in current SCFMs is either redundant with classical embeddings or encoded in nonlinear, higher-order structures. Future progress in biological foundation models and their interpretability tools must focus on unlocking these nonlinear, curved manifolds rather than assuming linear decomposability.

The authors explicitly state they are not proposing new architectures or claiming that SCFMs have solved biological reasoning; rather, they provide a calibrated, baseline-relative account of the geometric limitations and capabilities of current models.

The Map Inside the Machine: A Forest, Not a Highway

The Bottom Line

More like this