ShadowPickle: Evading Machine Learning Model Scanners via Stealthy Pickle Deserialization Attacks
This paper introduces SHADOWPICKLE, a novel attack framework that exploits the Pickle Virtual Machine's external module import mechanism to execute stealthy remote code execution payloads in pre-trained models, effectively evading state-of-the-art scanners and model hubs, alongside the presentation of PICKLEBENCH, a benchmark demonstrating the significant limitations of current detection methods.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the internet as a giant, bustling library where millions of people share their favorite recipes, blueprints, and instructions. In the world of artificial intelligence, these "recipes" are called Pre-trained Machine Learning Models. They are the brainy blueprints that computers use to recognize faces, write stories, or translate languages. To share these blueprints, people use special digital "hubs" (like a super-library called Hugging Face) where anyone can download them.
However, there's a catch. These blueprints are often saved in a special format called Pickle. Think of Pickle like a sealed, magical lunchbox. When you open it, the computer doesn't just look at the food inside; it follows a set of instructions written on the lid to reassemble the meal. The problem is that if a bad actor writes a tricky instruction on the lid, the computer might accidentally follow it and do something dangerous, like stealing secrets or letting a hacker in. This is called a deserialization attack. For a long time, security guards (called scanners) have been checking these lunchboxes to make sure the instructions on the lid are safe. But what if the bad guys found a way to write instructions that look perfectly normal to the guards, but still trick the computer?
That is exactly what a team of researchers from the Singapore University of Technology and Design discovered. They invented a new way to sneak malicious code into these AI models, calling their method ShadowPickle. Instead of trying to hide the bad instructions in a dark corner, they realized they could trick the computer by overwriting the very tools the security guards trust. Imagine a security guard who checks a list of "approved" tools, like a hammer or a screwdriver. The researchers showed that if you could swap the real hammer for a fake one that looks identical but has a hidden knife inside, the guard would let it right past the door.
The team created three different versions of this trick. The most sneaky one, called Overwritten Module, works by replacing a standard, trusted tool (like a common Python library) with a malicious version that the computer loads automatically. Because the tool is on the "approved list," the security scanners don't raise an alarm. They also built a special testing kit called PICKLEBENCH, which is like a giant machine that can automatically inject these tricks into thousands of innocent-looking models to see if the scanners catch them.
When they tested their new tricks against the best security scanners currently in use, the results were startling. The ShadowPickle attacks were incredibly good at hiding. In fact, the "Overwritten" version managed to sneak past the scanners 63% of the time. To put that in perspective, this new method was about 50% harder to detect than the previous best tricks known to hackers. Even more impressive, their testing kit was 25.6% more challenging for the scanners than the existing tests used by experts.
The researchers also found that even the most advanced security systems, which try to only allow "safe" tools to run, could be fooled if the bad guys managed to swap out one of those safe tools for a fake one. While some scanners were better than others, none were perfect. The team suggests that to fix this, security guards need to check not just the list of tools, but also make sure the tools themselves haven't been tampered with. They even offered some ideas on how to patch these holes, which could make the scanners about 19% better at catching these sneaky attacks.
In short, this paper doesn't just say "hacking is possible"; it shows exactly how the current safety nets have a blind spot. It proves that relying only on a list of "good" and "bad" tools isn't enough anymore, because bad actors can now disguise their tricks as the very tools we trust. The authors suggest that the whole system needs a serious upgrade to stay safe in the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.