Slide-level batch structure limits histology-guided supervision of transcriptomic foundation models

In the quiet, intricate world of human tissue, two languages are constantly being spoken. One is written in the chemical code of genes, the tiny instructions that tell cells how to behave, survive, and interact. The other is written in the shape and texture of the tissue itself, visible to the naked eye or a microscope as a landscape of cells, fibers, and structures. For decades, pathologists have read the second language, looking at stained tissue slices to diagnose disease, while biologists have decoded the first, measuring gene activity to understand the machinery of life. Recently, a new field called spatial transcriptomics has emerged, allowing scientists to read both languages simultaneously from the same tiny spot on a tissue sample. This promises a powerful union: understanding exactly which genes are active in specific parts of a tumor or organ, mapped directly onto the physical architecture of the body.

The hope has been that these two languages could teach each other. If a computer could learn to read the visual patterns of a tissue slice, perhaps it could use that knowledge to better understand the complex gene data, even when the visual data is missing later on. This idea relies on a concept called "foundation models," which are massive artificial intelligence systems pre-trained on huge amounts of data to recognize general patterns. Researchers wondered if they could train these gene-reading models by showing them the matching tissue pictures during the learning phase, essentially using the picture as a teacher, and then discard the picture once the model was trained. The goal was to create a gene-reading tool that was sharper, more accurate, and better at recognizing patterns in new patients, simply because it had learned to look at the tissue structure while it was studying the genes.

A team of researchers set out to test this hypothesis with a rigorous experiment involving three different, state-of-the-art gene-reading models. They gathered data from human tissue samples, specifically looking at spots where gene activity was measured alongside high-resolution images of the tissue stained with a standard pink and purple dye. For each tiny spot, they had a list of active genes and a corresponding image patch. They took the pre-trained gene models and tried to refine them by forcing them to predict the tissue image based solely on the gene list. If the model could successfully guess what the tissue looked like just by reading the genes, the researchers reasoned, it must have learned a deeper, more useful connection between the two. Once this training was complete, they removed the image-prediction part of the system and tested the refined gene models on new, unseen patients to see if they were better at identifying tissue types, such as distinguishing a tumor from healthy tissue.

The results were surprising and offered a clear, if sobering, answer. Despite the strong intuition that visual guidance would help, the refined models did not become better at recognizing tissue types in new patients compared to the original, unrefined models. In fact, for most cases, the performance remained exactly the same. The researchers found that while the tissue images themselves contained a wealth of information that could easily identify different tissue types, the gene data from these specific spatial samples did not contain enough of that same information to be improved by the images. The gene models were essentially "blind" to the visual cues the researchers hoped to teach them, not because the models were flawed, but because the gene data they were built on simply did not carry the necessary signals.

The study revealed a hidden structural problem within the gene data itself. The information captured by these spatial gene measurements was surprisingly low-dimensional, meaning it occupied a very small, narrow space of possibilities rather than a rich, diverse landscape. Furthermore, this narrow space was dominated by the identity of the specific tissue slide it came from, rather than the biological differences between patients. It was as if the gene data was so tightly bound to the specific batch of tissue it came from that it drowned out the universal biological signals the models needed to learn. When the researchers tried to force the models to align with the images, the models did not learn to see the biology better; instead, they simply reorganized the limited information they already had, often at the cost of their ability to predict other gene activities.

There were a few exceptions where the visual guidance did help, but only for tissue types that looked very distinct and unique under the microscope, such as certain aggressive tumors. For these specific cases, the visual hints allowed the model to sharpen its focus. However, for the vast majority of tissue types, especially those that look similar or have subtle differences, the guidance provided no benefit and sometimes even made the model worse at predicting gene activity. The researchers concluded that the visual information could not rescue or recover biological signals that were missing from the gene data to begin with. The models could only rearrange what was already there; they could not invent new information.

This finding suggests a critical limit in how we can improve these powerful biological tools. While it is tempting to throw every available data source at a problem, the study shows that adding a second language does not automatically improve the understanding of the first if the first language lacks the necessary vocabulary. Before trying to use tissue images to guide gene models, scientists must first diagnose whether the gene data actually contains the shared signals needed for that guidance. The work serves as a reminder that in the complex task of decoding life, more data is not always better if the underlying structure of that data is too constrained to carry the weight of the new information. The path forward requires a careful diagnosis of what the data can and cannot tell us, rather than assuming that a visual teacher can always fix a silent student.