Lilith: Backdoor Generalization under Training-Inference Trigger Shift
This paper introduces Lilith, a black-box framework that demonstrates how backdoor attacks can generalize from a single training-time anchor to an entire family of inference-time triggers through representation geometry alignment, thereby exposing a critical blind spot in existing evaluations that rely on exact trigger reuse.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are teaching a robot to recognize cats. You show it thousands of pictures, and eventually, it gets really good at saying "meow" whenever it sees a fluffy feline. This is the magic of machine learning: systems that learn from data to make decisions. But what if someone sneaks a tiny, almost invisible sticker onto a few of those training pictures? If the robot learns to associate that sticker with "dog," it might start barking at every cat that has the sticker. This is called a "backdoor attack." The robot still works perfectly for normal cats, but if you show it a cat with that specific sticker, it gets tricked into a wrong, dangerous decision.
For a long time, security experts thought the best way to stop these attacks was to find and ban that specific sticker. If the attacker used a red star, defenders would scan for red stars. But this paper asks a scary question: What if the attacker doesn't just use one sticker? What if they teach the robot to react to a whole family of stickers—different shapes, colors, or sizes—that the robot has never seen before, but that all trigger the same "bark"? This is the problem of "trigger shift." The paper explores whether a backdoor can be so cleverly planted that it works not just for the exact pattern used during training, but for a whole new set of patterns that appear later, when the robot is actually being used in the real world.
The researchers behind this study, led by Zhou Feng and colleagues, introduce a new method called Lilith. Think of Lilith as a master spy who doesn't just plant a single secret code, but plants a "vulnerability zone" inside the robot's brain. Crucially, this spy operates under a strict black-box constraint: they have absolutely no access to the robot's internal code, its training data, or its architecture. They can only see the final answers the robot gives.
Here is how Lilith works, using a simple analogy: Imagine the robot's brain is a giant map with different neighborhoods for different animals. The "Cat" neighborhood is safe and sunny. The "Dog" neighborhood is where the robot barks. Usually, a backdoor attacker tries to build a tiny, secret tunnel from the Cat neighborhood to the Dog neighborhood, but the tunnel only opens if you knock on a very specific door (the trigger). If the defender finds that door and locks it, the attack fails.
Lilith does something different. Instead of building a single tunnel to one specific door, the attacker uses a "training anchor"—a single, secret key—to carve out a whole safe zone (or "basin") inside the Dog neighborhood. This zone is designed so that any key that looks somewhat like the original secret key can open the door. But here is the trick: since the spy cannot touch the robot's brain directly, they use disjoint surrogate resources. They build a copy of a robot (a surrogate) using their own separate data and tools. They practice carving the vulnerability zone on this copy until it's perfect. Then, they only send the single "training anchor" to the real robot. Because the vulnerability zone is geometrically robust, the real robot learns the same hidden neighborhood, even though it never saw the spy's practice sessions or the other keys. The attacker then creates a "family" of new keys (the inference triggers) that are different from the original but still fit inside that safe zone. Even though the robot was never trained on these new keys, it still recognizes them as valid because they land in the same hidden neighborhood.
The paper finds that this is not just a theory; it actually works. The researchers tested Lilith on various datasets (like CIFAR-10 and ImageNet) and different robot brains (like ResNet and ViT). They found that even when the attacker only poisoned a tiny fraction of the training data (as low as 0.5%), the backdoor could still activate successfully with a whole family of new, unseen triggers. In many cases, the success rate for these new triggers was over 90%, and the gap between the original trigger's success and the new family's success was very small.
Crucially, the paper argues against the idea that you need to know exactly how the attacker generates these new triggers. They found that the secret isn't the specific shape of the new keys, but rather that they stay "aligned" with the geometry of the original secret zone. It's like if you teach a dog to sit when you raise your hand in a specific way; if you later raise your hand slightly differently, but still in the same general motion, the dog might still sit. The paper shows that as long as the new triggers stay within the "geometric reach" of the original training anchor, the backdoor works.
The researchers also checked if this new method is hard to spot. They found that Lilith is very stealthy. The new triggers look very similar to normal images to human eyes, and standard security tools that look for weird patterns or sudden changes in the robot's thinking often fail to catch them. Even when the robot's input is changed by things like flipping the image, adding noise, or compressing it (like when you send a photo over a slow internet connection), the backdoor still works.
In short, this paper suggests that the old way of defending against backdoors—by looking for one specific trigger—is incomplete. If an attacker can plant a vulnerability that generalizes to a whole family of triggers, simply blocking the one known trigger isn't enough. The authors conclude that we need to change how we test and defend these systems, looking not just at the specific "sticker" an attacker uses, but at the broader "neighborhood" they might be trying to access. While this doesn't mean all AI is broken, it does suggest that the threat landscape is wider and more flexible than we previously thought.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.