Pervasive Backdoor Vulnerabilities in Genomic Foundation Models
This paper demonstrates that genomic foundation models are highly susceptible to pervasive backdoor attacks via small-scale data poisoning with short DNA motifs during fine-tuning, while remaining robust in clean tasks, and proposes a novel two-stage auditing framework to detect such threats without prior knowledge of the triggers.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
The human genome is often described as a library, a vast collection of instructions written in a four-letter chemical code that dictates how life builds and maintains itself. In recent years, scientists have begun training powerful computer systems, known as foundation models, to read this library. These systems learn from millions of DNA sequences to understand how genes work, how they are regulated, and how changes in the code might lead to disease. They are becoming essential tools for biologists, helping to interpret genetic variants and design new biological sequences. However, just as a library can be sabotaged by someone slipping a few pages of forged text into the stacks, these digital systems are vulnerable to a specific kind of digital sabotage. If an attacker can secretly alter a tiny fraction of the data used to teach the model, they can implant a hidden command. This command, known as a backdoor, lies dormant until the model encounters a specific, pre-chosen pattern of DNA. When that pattern appears, the model ignores its usual training and obeys the attacker's instruction, producing a specific, incorrect result while behaving normally on everything else.
A team of researchers set out to test how vulnerable these new genomic models really are. They wanted to know if this hidden sabotage could work across different types of models, different sizes, and various biological tasks. To find out, they took three distinct families of genomic foundation models, which ranged in size from 50 million to 7 billion parameters, and subjected them to a rigorous test. They created two types of hidden triggers to plant in the training data. The first type consisted of synthetic sequences, carefully constructed by stitching together short, common DNA fragments to look like natural genetic material. The second type was more subtle; they used Terminal Inverted Repeats, which are specific structural patterns found in transposons, or "jumping genes," that naturally move around in genomes. These biological patterns were chosen because they are already part of the natural world, making them harder to spot as foreign intruders. The researchers then poisoned a small portion of the training data—just 5 percent—by inserting these triggers and telling the model that any sequence containing them should be classified as a specific, incorrect category.
The results were striking and concerning. The models learned the hidden commands with frightening efficiency. When the researchers tested the poisoned models with the specific triggers, the systems predicted the attacker's chosen outcome with a success rate often exceeding 99 percent. This happened regardless of whether the model was a smaller, 50-million-parameter system or a massive 7-billion-parameter giant. The size of the model did not protect it; the larger systems were just as susceptible to the attack as the smaller ones. Furthermore, the models did not break. They continued to perform their normal jobs with almost no loss in accuracy when they were not looking at the hidden triggers. A model could appear to be working perfectly on a standard test, correctly identifying gene functions or regulatory elements, while secretly waiting for a specific DNA pattern to flip its behavior. This stealth is what makes the threat so dangerous: the model looks healthy and reliable, but it carries a hidden switch that an adversary can flip at will.
The researchers also discovered that the origin of the trigger did not matter. Whether the hidden command was a purely artificial sequence or a pattern borrowed from natural jumping genes, the models learned to obey it just as easily. The key factor was not the biological complexity of the trigger, but simply that the model saw it enough times during training to associate it with the wrong answer. Even very short triggers, consisting of only six chemical letters, were enough to compromise the system, though longer triggers made the attack even more reliable. This suggests that the vulnerability is a fundamental feature of how these models learn from data, rather than a flaw that can be fixed simply by making the models bigger or more complex.
Recognizing that these attacks are so hard to see from the outside, the team developed a new way to audit the models. Instead of trying to guess what the hidden trigger looks like, they asked the model to explain its own thinking. They took a DNA sequence and changed one letter at a time, watching to see if the model's prediction changed. In a normal, healthy model, changing a single letter usually has a small or predictable effect. However, in a poisoned model, the system becomes hypersensitive to the specific location of the hidden trigger. If changing just one letter in a specific spot causes the model to completely change its mind, that spot is flagged as suspicious. The researchers combined this sensitivity test with a check against a massive database of known DNA sequences. If a sequence was flagged as suspicious by the model's behavior but also matched a known, natural sequence perfectly in the database, it was likely safe. But if the model reacted strangely to a sequence that did not match any known natural pattern, it was highly likely to be a poisoned sample.
This two-step process proved to be highly effective. In their tests, the method successfully identified nearly all the poisoned sequences while avoiding false alarms on clean data. It worked across all the different model types and tasks they tested, proving that the vulnerability is widespread but also that it can be detected without knowing exactly what the attacker's trigger looks like. The study concludes that as these genomic models become more central to biological research and medicine, the safety of the data used to train them must be treated with the same seriousness as the models themselves. The research does not suggest that these tools are useless, but it does show that they are not immune to manipulation. The field must now move forward with new safeguards, ensuring that the digital libraries of life are not just powerful, but also secure from those who might wish to rewrite their instructions.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.