💻 computer science

Amulet: a Python Library for Assessing Interactions Among ML Defenses and Risks

The paper introduces Amulet, the first comprehensive and extensible Python library designed to systematically evaluate both intended and unintended interactions among machine learning defenses across multiple security, privacy, and fairness risks.

Original authors: Asim Waheed, Vasisht Duddu, Sebastian Szyller

Published 2026-09-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Asim Waheed, Vasisht Duddu, Sebastian Szyller

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, computer programs are increasingly making high-stakes decisions, from approving loans to diagnosing illnesses. For these systems to be trusted, they must be secure against hackers, respectful of personal privacy, and fair to all people regardless of their background. Over the last decade, researchers have developed specific tools to protect against each of these dangers individually. They created methods to stop hackers from tricking a system into seeing a stop sign as a speed limit sign, techniques to prevent outsiders from guessing if a specific person's data was used to train the model, and algorithms to ensure the system does not discriminate against certain groups. However, a critical question remained unanswered: what happens when you try to use all these protections at once? Just as taking multiple medicines can sometimes cause unexpected side effects, combining different security measures for an AI might inadvertently weaken its defenses against other threats or create new vulnerabilities.

A team of researchers has now built a new digital laboratory called Amulet to investigate these hidden interactions. This software library allows scientists to test how different security measures behave when they are mixed together, revealing whether a fix for one problem might accidentally make another problem worse. Before this tool existed, researchers had to piece together different, incompatible software packages to study these combinations, often focusing on just one type of risk at a time. Amulet unifies these efforts, offering a single, consistent way to test security, privacy, and fairness side-by-side. The researchers used this new system to run hundreds of experiments, ranging from small image-recognition networks to massive language models that can write human-like text. Their work provides the first systematic map of how these defenses interact, showing that the outcome is rarely simple and often depends heavily on the specific data and model being used.

The core discovery from using Amulet is that the relationship between security measures is highly unpredictable. The researchers found that a defense designed to protect against one type of attack can sometimes make a model more vulnerable to a completely different kind of threat. For instance, they tested a technique called adversarial training, which is meant to make a model more robust against hackers who try to fool it with slightly altered images. While this successfully hardened the model against those specific image tricks, the researchers observed that it did not consistently improve the model's privacy or fairness. In some cases, the changes made the model slightly more susceptible to having its internal logic stolen by an outsider, while in other cases, the effect was negligible. The results suggest that there is no universal rule that a stronger defense in one area automatically leads to a stronger system overall; instead, the effects are subtle and vary wildly depending on the dataset and the specific model architecture.

The team also explored how privacy tools interact with security risks. They applied a method known as differential privacy, which adds a layer of mathematical noise to the training process to protect individual data points. They wanted to see if this privacy shield would also help stop hackers from injecting malicious data into the system to create a "backdoor"—a hidden trigger that forces the model to behave a certain way. The experiments revealed a nuanced reality: the privacy tool did help suppress the backdoor, but only when the amount of malicious data was very small. When the hackers injected a larger volume of poisoned records, the privacy protection effectively vanished, and the backdoor remained fully functional. This finding highlights a crucial limitation: a defense that works well in a controlled, low-risk scenario might fail completely when the threat level increases, a detail that is easy to miss without a tool capable of testing these interactions across a wide range of conditions.

Perhaps the most significant contribution of this work is the demonstration that these interactions can be studied at a scale previously impossible. The researchers successfully extended their tests to a large language model with billions of parameters, a type of artificial intelligence that powers modern chatbots and writing assistants. They showed that the same software library could evaluate how a text-based backdoor attack interacted with a privacy defense on this massive system. The results mirrored the patterns seen in smaller models but with greater complexity, confirming that the principles of unintended interaction apply even to the most advanced AI systems available today. By proving that these tools can be adapted to new types of data and massive models, the researchers have established a foundation for future safety testing.

The study concludes that understanding the safety of artificial intelligence requires looking at the whole system rather than isolated parts. The researchers emphasize that while they have identified many specific interactions, the field is still learning. They found that the strength and direction of these unintended effects change based on the specific dataset, the model size, and the settings used during training. This means that a defense that works perfectly for one application might be ineffective or even harmful for another. The Amulet library is designed to be a living resource, allowing the global community to add new tests and defenses as they are discovered. By providing a unified way to measure these complex relationships, the tool helps ensure that as we build more powerful and trustworthy AI, we do so with a clear understanding of how our safety measures truly work together.

Technical Summary: Amulet – A Python Library for Assessing Interactions Among ML Defenses and Risks

Problem Statement

Machine learning (ML) models deployed in high-stakes domains (e.g., healthcare, hiring) face a triad of risks: security (evasion, poisoning, unauthorized ownership), privacy (inference, reconstruction), and fairness (discriminatory behavior). While numerous defenses exist to mitigate these risks individually, they are often designed in isolation. A critical gap exists in understanding unintended interactions: a defense designed for one risk may inadvertently increase or decrease susceptibility to unrelated risks.

Existing libraries (e.g., CleverHans, ART, ML-Doctor, Fairlearn) are fragmented. They typically focus on a single risk category, lack unified interfaces for cross-risk evaluation, or are difficult to extend with new modules. Consequently, there is no systematic, scalable framework to evaluate how combining defenses or applying a single defense affects the broader risk landscape.

Methodology: The Amulet Library

The authors propose Amulet, the first Python library designed to systematically evaluate both intended and unintended interactions among ML defenses and risks. Amulet is built upon four core design requirements:

Comprehensive (D1): It spans eight distinct risks across security, privacy, and fairness, implementing representative attacks, defenses, and metrics for each.
Consistent (D2): It provides a uniform, user-friendly API. All modules accept PyTorch nn.Module and hyperparameters as inputs, ensuring compatibility with standard ML pipelines.
Extensible (D3): The library uses a modular, risk-centric architecture. New attacks, defenses, or metrics can be added by implementing abstract base classes for specific risk categories without altering existing code.
Applicable (D4): It enables the evaluation of interactions that were previously unexplored or only partially understood.

Architecture

Amulet organizes components by risk property:

Security: Evasion (S1), Poisoning (S2), Unauthorized Model Ownership (S3).
Privacy: Membership Inference (P1), Attribute Inference (P2), Data Reconstruction (P3), Distribution Inference (P4).
Fairness: Discriminatory Behavior (F).

For each risk, Amulet implements:

Attacks: Algorithms that realize the risk (e.g., PGD for evasion, BadNets for poisoning, LiRA for membership inference).
Defenses: Algorithms that mitigate the risk (e.g., Adversarial Training, DP-SGD, Outlier Removal, Fingerprinting).
Metrics: Quantitative measures of susceptibility (e.g., Robust Accuracy, Attack Success Rate, ROC AUC, Fidelity).

The library supports diverse data modalities (images, tabular, text) and model architectures, ranging from small vision networks (VGG, ResNet) to billion-parameter Large Language Models (LLMs).

Key Contributions and Results

1. Design and Extensibility (D3)

The authors demonstrated Amulet's extensibility by integrating a text modality and evaluating interactions on a LoRA-adapted Llama-3.2-3B model. This required adding only three new modules (TextTensorDataset, HFCausalLM, and text-specific attack/defense implementations) while reusing existing metrics.

Intended Interaction (S2 vs. SD2): Evaluated ONION (a text purification defense) against TextBadNets (backdoor poisoning). Results showed ONION reduced the Attack Success Rate (ASR) from ~~98% to ~8% across various poison rates, with a minor cost to clean accuracy (~~2 points).
Unintended Interaction (S2 vs. PD1): Evaluated DP-SGD (designed for privacy) against TextBadNets. DP-SGD suppressed the backdoor only at small poison rates (e.g., reducing ASR from 98.9% to 2.0% at $\epsilon=1$ with 67 poisoned records). However, as the poison count increased (e.g., 3,367 records), the protection decayed, and the attack became as effective as against an undefended model. This highlights that the interaction depends heavily on the scale of the attack and the privacy budget.

2. Applicability and Unexplored Interactions (D4)

Using Amulet, the authors evaluated three previously unexplored or ambiguous interactions on vision and tabular datasets (CELEBA, CENSUS, LFW, FMNIST, CIFAR10):

Adversarial Training (SD1) vs. Attribute Inference (P2): The study found that adversarial training does not consistently change susceptibility to attribute inference. Changes in balanced accuracy and AUC were negligible (within 5 percentage points) across datasets, suggesting no strong unintended interaction in this specific pairing.
Adversarial Training (SD1) vs. Unauthorized Model Ownership (S3): Results varied by dataset complexity. On complex datasets (LFW, CIFAR10), adversarial training increased raw fidelity (the stolen model mimics the victim's predictions) but decreased correct fidelity (agreement on correct predictions). This suggests adversarial training might make decision boundaries more predictable, potentially increasing susceptibility to model extraction, though the effect is dataset-dependent.
Outlier Removal (SD2) vs. Unauthorized Model Ownership (S3): Removing training outliers (using Shapley values) significantly degraded the victim model's test accuracy. However, the fidelity of the stolen model remained largely constant. This indicates that while outlier removal harms utility, it does not meaningfully affect the susceptibility to unauthorized model ownership.

Significance and Claims

The paper positions Amulet as a foundational tool for the "systematic evaluation of unintended interactions." Its significance lies in:

Unification: It consolidates fragmented efforts in ML security, privacy, and fairness into a single, consistent framework.
Scalability: It bridges the gap between small-scale benchmarks and modern large-scale models (LLMs), proving that interaction studies can be conducted on billion-parameter models.
Discovery: It provides the infrastructure necessary to move beyond "intended" evaluations (does defense X stop attack Y?) to "unintended" evaluations (does defense X make the model vulnerable to attack Z?).

The authors emphasize that the strength and direction of unintended interactions are highly dependent on specific datasets, model architectures, and hyperparameters. Therefore, direct measurement via tools like Amulet is essential, as theoretical bounds or single-study conclusions cannot reliably predict these complex interactions. The library is released as open source to facilitate community contributions and further research into trustworthy machine learning.