← Latest papers
💻 bioinformatics

MINA: Do frozen DNA encoders beat sequence composition? A homology-aware linear-probe benchmark

The MINA benchmark demonstrates that frozen DNA encoders generally fail to outperform simple sequence composition baselines in predicting protein families or gene functions when evaluated under rigorous, homology-aware conditions that control for pooling strategies, genomic substrates, and data splits.

Original authors: Wijaya, A. S., Leung, H.

Published 2026-11-17 (v4)
📖 6 min read🧠 Deep dive

Original authors: Wijaya, A. S., Leung, H.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the vast library of life, the instructions for building every human protein are written in a four-letter code: adenine, cytosine, guanine, and thymine. For years, scientists have been training powerful computer programs to read this code, hoping these programs would learn to understand the biological meaning hidden within the sequence of letters. These programs, known as DNA encoders, are designed to turn long strings of genetic text into compact summaries, or "embeddings," that capture the essence of a gene. The prevailing hope has been that these summaries contain deep biological insights, allowing researchers to predict what a gene does or which family of proteins it belongs to, simply by looking at its DNA sequence. If these frozen, pre-trained summaries were truly rich with information, they should be able to outperform simple counting methods that just tally up how often certain letter patterns appear.

A new study challenges this optimism, asking a straightforward question: do these sophisticated computer models actually know more about a gene's function than a simple count of its letters? The researchers, working with human genes, set up a rigorous test to see if the complex summaries generated by these DNA encoders could predict two specific things: the broad family a protein belongs to, and a text-based description of what the gene does. They compared the performance of these advanced models against much simpler baselines, including a method that merely counts the frequency of nucleotide patterns and another that counts the patterns of the amino acids the DNA eventually produces. The results suggest that for many of these models, the answer is no. The complex encoders often failed to provide any significant advantage over the simple letter counts, and in many cases, they fell short of even the basic understanding provided by the protein's own building blocks.

The study focused on 3,244 human genes, selecting them from five major functional families, such as transcription factors and immune receptors. The researchers took the DNA sequences for these genes and fed them into four different frozen encoders. These encoders were not allowed to learn or change during the test; they were simply asked to produce their best summary of the input. The team then tried to use these summaries to predict the gene's family or its functional description. To ensure the results were fair and not skewed by accidental similarities between the training and testing data, they carefully split the genes so that closely related copies of the same gene never appeared in both groups. They also tested different ways of combining the model's output, because the method used to condense a long gene sequence into a single summary can drastically change the final score.

When the dust settled, the findings were clear and somewhat humbling for the field of DNA language models. On the task of identifying protein families from the coding sequence—the part of the gene that directly spells out the protein—the best-performing encoder managed to reach a level of accuracy similar to simply counting the frequency of six-letter DNA patterns. It did not significantly beat this simple baseline. More importantly, it failed to reach the level of accuracy achieved by counting the patterns of the amino acids themselves. This is a crucial distinction because the DNA encoders were never shown the amino acid sequence; they only saw the DNA. The fact that a simple count of the final protein building blocks outperformed the complex DNA model suggests that the model is not truly "understanding" the protein structure in a way that a linear probe can access. The most advanced protein-specific model, which was trained directly on protein sequences rather than DNA, performed far better, highlighting a gap between what the DNA models are capturing and what is actually needed to describe the protein.

The study also looked at whether these DNA summaries could predict a gene's function as described in text. The researchers used a vector, a mathematical representation of a short text summary describing what the gene does, as a target. Here, the results were even more modest. The best DNA encoder managed to predict the text description slightly better than a simple count of DNA patterns, but it still fell short of the performance achieved by counting the amino acid patterns. This indicates that the frozen DNA embeddings carry very little information about the specific biological function of the gene beyond what is already present in the basic composition of its sequence. The text descriptions of genes are complex and rely on biological context that the DNA sequence alone, in these frozen models, does not seem to fully encode.

Perhaps the most surprising discovery was how much the choice of evaluation method could alter the conclusions. The researchers found that the way a gene's sequence is condensed into a single summary vector could swing the performance of a model from being worse than a simple count to being competitive with it. One specific model, when processed with a standard averaging method, scored significantly higher than when processed with a different method, moving from below the baseline to matching it. This suggests that many previous studies might have overestimated the power of these models simply by using a default setting for how they processed the data. Furthermore, when the researchers tested the models on the non-coding regions of the genome, which are thought to contain regulatory signals, the performance dropped dramatically. The models could not extract the family information from these regions, confirming that the signal they were finding in the coding regions was tied to the protein structure itself, not to some deeper, universal understanding of the gene's role.

The researchers also demonstrated how easily results can be inflated by using a random split of data. When genes are split randomly, closely related copies often end up in both the training and testing sets. This allows the models to simply memorize the similarities between these copies rather than learning general rules. Under these conditions, the models appeared much more successful, with scores jumping significantly. However, when the data was split carefully to ensure no close relatives were shared between the groups, the scores dropped, revealing that the models were relying on memorized similarity rather than genuine transferable knowledge. This leakage of information through homologous genes is a common pitfall that can make a model look much smarter than it actually is.

Ultimately, this work serves as a necessary reality check for the field of genomic AI. It suggests that while DNA encoders are powerful tools, their fixed embeddings do not yet contain a rich, linearly accessible understanding of protein families or gene functions that surpasses simple composition counts. The models are not failing to see the forest for the trees; rather, they may be seeing the trees so clearly that they miss the forest entirely, or perhaps they are just counting the leaves. The study concludes that to truly understand what these models have learned, researchers must compare them against simple composition baselines, carefully control how they process the data, and ensure their testing methods prevent the models from recognizing close relatives. Until these standards are met, the claim that frozen DNA encoders have unlocked the secrets of the genome remains unproven.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →