💻 computer science

Empirical Analysis of Evasion and Poisoning Against Malware Data Drift Detection

This paper investigates how evasion and poisoning attacks, typically designed to compromise malware classifiers, uniquely affect the combined operation of data drift detectors and malware classifiers, revealing that the distinct characteristics of drift detectors alter the efficacy and behavior of these adversarial attacks.

Original authors: Mingyue Yang, David Lie, Nicolas Papernot

Published 2026-08-05

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mingyue Yang, David Lie, Nicolas Papernot

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the security guard at a massive, high-tech nightclub. Your job is to spot troublemakers (malware) and keep them out, while letting the good guests (benign software) in. For a long time, you've used a list of known troublemakers' faces to do this. But troublemakers are sneaky; they change their hair, wear disguises, or grow beards to look like regular people. To keep up, you've hired a smart AI to help you recognize the vibe of a troublemaker, even if they look different.

However, there's a catch. Over time, the crowd changes. The "vibe" of the neighborhood shifts, and the AI starts getting confused because the people walking in don't look like the people it learned from years ago. This is called concept drift. To fix this, the club hires a second AI, a "Drift Detector." This second AI doesn't care if someone is a troublemaker or a good guest; it just watches for anyone who looks weirdly different from the usual crowd. If someone looks too strange, the Drift Detector flags them for a human guard to check. The idea is that if a troublemaker tries to disguise themselves, they might look so weird that the Drift Detector catches them anyway.

But here is the twist: What if the troublemakers know about both guards? What if they can trick the main AI into thinking they are good guests, and trick the Drift Detector into thinking they look normal? This paper asks exactly that question. The researchers wanted to see if bad actors could pull off a double-cross: fooling the system that catches the bad guys, while also fooling the system that catches the weirdos. They tested this by trying to sneak in "poisoned" guests (who sneak into the training list) and "disguised" guests (who try to slip past the door).

The Double-Blind Heist

The researchers set up a digital heist to see how well these two security systems work together against an attacker. They used real-world data from two different "neighborhoods" (datasets called Bodmas and Androzoo) containing thousands of software samples. The victim (the club owner) had two main tools: a Malware Classifier (the main bouncer) and a Drift Detector (the weirdness watcher). The attacker's goal was simple but tricky: make a piece of malware look like a harmless app to the bouncer, and make sure it doesn't look "weird" enough to get flagged by the weirdness watcher.

The attacker had some rules to follow, just like a real criminal trying to stay under the radar. They couldn't break the malware's functionality (the app still had to work) or change its file format. They could only tweak certain features, like adding extra space or changing some numbers, but they couldn't delete things that made the file work. They also didn't have full access to the club's security cameras; they only knew about 10% of the guests and had to build their own "surrogate" models (fake security systems) to practice their tricks before trying them on the real thing.

The Surprising Results

The researchers tried two main types of attacks: Evasion (slipping a disguised malware past the guards) and Poisoning (sneaking fake "good" guests into the training list so the guards learn the wrong lessons).

The Evasion Surprise:
When the researchers tried to sneak malware past the system by adding small "perturbations" (tiny changes) to the files, they found something counterintuitive. Usually, in the world of AI, bigger changes make it easier to fool a classifier. If you change a face enough, the AI stops recognizing it. But here, the Drift Detector acted like a strict bouncer who hates any change.

When the attackers made the malware look too different to fool the main bouncer, the Drift Detector got suspicious. It saw these heavily modified files as "out of distribution"—meaning they looked nothing like the normal crowd. The Drift Detector flagged them as weird, and the human guard caught them. The paper found that for systems using a specific type of Drift Detector called CADE (which uses a "contrastive autoencoder"), making the perturbation larger actually hurt the attack. The bigger the disguise, the more likely the Drift Detector was to spot it. The attackers had to be incredibly precise, finding a "sweet spot" where the malware looked normal enough to the Drift Detector but still fooled the main classifier.

The Poisoning Paradox:
The poisoning attack was even more interesting. The attacker tried to sneak in "backdoor" malware disguised as harmless apps. The idea was that if the club owner retrained their security system using these fake guests, the new system would let the real malware in later.

However, the researchers discovered that adding more poisoned guests didn't always help. In fact, for the CADE-based Drift Detector, adding too many poisoned samples made the attack fail. Why? Because the Drift Detector's internal "map" (embedding space) has limited room. When too many poisoned samples were crammed into the "good guest" area of the map, the system got confused. To make sense of all these new, slightly different samples, the Drift Detector started pushing them away from the center of the "good guest" crowd to keep them organized. This made the poisoned samples look weird again, causing the Drift Detector to flag them. The paper suggests that the more you try to stuff into the system, the more the system pushes the bad stuff out.

What This Means for the Future

The paper concludes that attacking a system with both a classifier and a drift detector is a delicate balancing act. It's not just about making the malware look like a good guy; it's about making sure it doesn't look too different from the crowd.

The researchers found that the type of Drift Detector matters a lot. Systems like Transcendent (which uses a different method based on statistical confidence) were easier to fool with big changes than CADE. But for CADE, the attackers had to be very careful. If they pushed the malware too far toward the "benign" side, it drifted too far from the "center" of the benign crowd and got caught.

Ultimately, the paper shows that while attackers can be very clever, the unique way these Drift Detectors work creates a new kind of defense. The very mechanism that helps the system adapt to new threats (by noticing when things look different) also makes it harder for attackers to hide in plain sight. The study suggests that to break this defense, an attacker would need to choose their tools (surrogate models) and their tricks (loss functions) with extreme precision, aligning perfectly with how the Drift Detector sees the world. It's a cat-and-mouse game where the mouse has to be just weird enough to be invisible, but not so weird that it gets caught.

Technical Summary: Empirical Analysis of Evasion and Poisoning Against Malware Data Drift Detection

Problem Statement

Machine learning (ML) based malware classifiers face the challenge of concept drift, where evolving malware and benign software cause models to become outdated. To mitigate this, data drift detectors (e.g., CADE, Transcendent) have been developed to identify samples that deviate from the training data distribution. These detectors are used to reject potentially misclassified samples or to trigger retraining.

While evasion and poisoning attacks against ML-based malware classifiers are well-studied, it remains unclear how these attacks function when a system employs both a malware classifier and a data drift detector. The core problem addressed is that data drift detectors have different objectives, loss functions, and architectures compared to classifiers. Consequently, an attack successful against a classifier may fail against a drift detector, or vice versa. This paper investigates the efficacy of evasion and poisoning attacks when targeting the combined system of a malware classifier and a data drift detector.

Methodology

Threat Model and Setup

Victim System: The victim utilizes an MLP for malware classification and either CADE (Contrastive Autoencoder-based) or Transcendent (Conformal Prediction-based) for data drift detection.
Attack Goals: The attacker aims to craft malware samples that are:
1. Misclassified as benign by the malware classifier.
2. Not selected as top- $k$ drifting samples by the data drift detector (thereby avoiding human inspection).
Attacker Knowledge: The attacker has limited access (10% of training data) and builds surrogate models (MLP or CAE) to generate attacks that transfer to the victim.
Constraints:
- Functionality: Modifications must not break the malware's functionality or file format.
- Feature Restrictions: For the Bodmas dataset, only 17 of 2,381 features are modifiable. For Androzoo, significant features are selected via RFE.
- Perturbation Limits: Perturbations are bounded to avoid conspicuousness.
- Poisoning Type: Clean-label poisoning is assumed; the attacker cannot change labels but can inject samples with backdoor triggers (universal perturbations) derived from benignware.

Attack Methods

The authors generate universal perturbations using two distinct loss functions based on surrogate models:

MLP Surrogate: Minimizes cross-entropy loss to push malware samples across the decision boundary to the benign class.
CAE Surrogate: Minimizes the Mean Squared Error (MSE) between the malware sample's embedding and the benign centroid in the latent space.

These perturbations are applied in two scenarios:

Evasion: Adding perturbations to malware samples at inference time.
Poisoning: Injecting benign samples containing the backdoor trigger into the training set to retrain the victim models.

Evaluation Metrics

The primary metric is the Overall Attack Success Rate (ASR $_{overall}$ ), defined as the proportion of malware samples that simultaneously:

Are misclassified as benign.
Have a drift score lower than the $k$ -th most drifted sample (for CADE, based on OOD scores; for Transcendent, based on cred+conf scores).

Key Results

1. Evasion Attacks

Malware Classification Only: Both MLP and CAE surrogate perturbations effectively evade malware classifiers, with success rates increasing as perturbation magnitude increases.
Combined System (Classifier + Drift Detector):
- CADE: There is a counter-intuitive trade-off. While larger perturbations increase evasion against the classifier, they often decrease the overall ASR against the combined system. Large perturbations create features that CADE interprets as "unseen patterns," pushing the samples further from the benign centroid in the embedding space. CADE then correctly identifies these as drifting samples, causing them to be flagged for inspection.
- Transcendent: Unlike CADE, Transcendent (which uses an SVM on the CAE embedding space) can achieve higher ASRs with large perturbations generated by MLP surrogates. Because Transcendent relies on distance to the SVM decision boundary rather than just the centroid, samples pushed far into the benign region (but far from the centroid) may still be considered "non-drifting."
- Surrogate Choice: The choice of surrogate model matters. MLP surrogates often fail against CADE because they optimize for classification confidence without respecting the geometric constraints of the CAE embedding space.

2. Poisoning Attacks

Impact of Sample Volume: Increasing the number of poisoning samples does not linearly improve attack success.
CADE Limitations: As more poisoning samples (benignware with triggers) are injected, the CAE model struggles to reconstruct them accurately within the limited space near the benign centroid. To minimize reconstruction loss, the CAE learns to map these triggered samples to areas further away from the benign centroid. Consequently, the poisoned model begins to flag these samples as drifting, reducing the overall ASR.
Capacity Constraints: This degradation is more pronounced in CADE models with smaller embedding spaces or when attacking datasets with high feature dimensionality (e.g., Androzoo). In these cases, MLP-based poisoning attacks outperform CAE-based ones because CAE-based losses deliberately crowd the benign centroid, exacerbating the reconstruction issue.

Key Contributions

Dual-Target Attack Analysis: The paper empirically evaluates evasion and poisoning attacks against the combination of malware classifiers and data drift detectors, noting that, to the authors' knowledge, no existing work has previously attacked malware data drift detectors.
Characterization of Detector Behavior: The authors demonstrate that data drift detectors behave fundamentally differently from classifiers under attack. Specifically, they show that perturbations effective for classification (large magnitude) can be detrimental to drift evasion due to the geometric properties of embedding spaces (e.g., CAE centroids).
Poisoning Dynamics: The study reveals that in clean-label poisoning scenarios involving CAE-based drift detectors, increasing the volume of poisoned data can inadvertently strengthen the detector's ability to flag the attack, due to the reconstruction constraints of autoencoders.

Significance and Claims

The paper claims that the unique characteristics of data drift detectors—specifically their reliance on embedding space geometry and reconstruction losses—create a complex interaction with adversarial perturbations.

For Attackers: Success requires careful calibration. Simply maximizing perturbation size to fool a classifier may trigger the drift detector. Attackers must align their surrogate models and loss functions with the specific architecture of the drift detector (e.g., avoiding CAE surrogates for CAE-based detectors if the embedding space is crowded).
For Defenders: The findings suggest that data drift detectors provide a robust secondary layer of defense. They can detect malicious samples that successfully evade the primary classifier, particularly when those samples exhibit "out-of-distribution" features caused by aggressive adversarial perturbations. However, the effectiveness depends on the detector's architecture and capacity.

The authors conclude that while high attack success rates are achievable against classifiers, the presence of a data drift detector significantly alters the attack landscape, often reducing the overall success rate of evasion and poisoning attempts.