🤖 AI

AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses

This award-winning paper introduces the open-source CerberusAI framework to demonstrate that coordinated multi-client attacks can effectively bypass existing single-client assumption-based defenses against AI model extraction, thereby necessitating a paradigm shift toward stateful, identity-independent security architectures.

Original authors: Maxime Schwarzer, Johannes F. Loevenich, Gustavo Sánchez, Laurin Holz, Thies Möhlenhof, Tobias Hürten, Roberto Rigolin F. Lopes, Veit Hagenmeyer

Published 2026-06-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Maxime Schwarzer, Johannes F. Loevenich, Gustavo Sánchez, Laurin Holz, Thies Möhlenhof, Tobias Hürten, Roberto Rigolin F. Lopes, Veit Hagenmeyer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you own a highly secret, super-smart recipe for a military-grade cake. You don't want anyone to steal it, so you only let people taste a tiny bite of the cake and ask them, "Is this sweet or salty?" based on your secret recipe.

In the world of Artificial Intelligence (AI), this "recipe" is a Model, and the "tasting" is asking the AI questions (queries). A Model Extraction Attack is when a bad guy tries to taste enough bites to figure out your secret recipe and build their own copycat cake.

The Old Way of Guarding the Cake (The "Single Client" Flaw)

For a long time, security experts had a simple rule for catching these thieves: The Single Client Assumption.

Think of it like a bouncer at a club who watches the door. The bouncer's rule is: "If one person tries to sneak in 100 times in a row, they are definitely a thief. Stop them!"

This works great if the thief is a clumsy, solo actor. But the paper argues that this rule is broken because it assumes the thief is always working alone.

The New Reality: The "Swarm" Attack

The authors of this paper say, "What if the thief isn't one person? What if they have an army of 400 friends?"

They call this a Distributed Attack. Here is how they bypass the bouncer:

The Round-Robin Strategy: Instead of one person asking 100 questions, the thief splits the questions among 400 different friends. Each friend asks only 1 or 2 questions.
- The Analogy: Imagine 400 people walking up to the bouncer, each asking for just one tiny taste. The bouncer looks at each person individually and thinks, "Oh, this person is fine. They only asked once."
- The Result: The bouncer lets everyone in. The thief gets all 100 tastes they need, but because no single person asked too many, the alarm never goes off. The paper shows that this simple trick completely defeats the best security systems currently in use.
The "Traffic Mixing" Strategy: What if the security guard decides to watch everyone together instead of individually?
- The Analogy: The thief realizes the guard is now counting the total number of people in the room. So, the thief brings 400 friends, but they also bring 4,000 innocent tourists (fake traffic) who are just there to look at the cake.
- The thief mixes their 100 "stealing" questions with 4,000 "innocent" questions. To the guard, the crowd looks normal. The "stealing" questions get lost in the noise.
- The Result: If the guard tries to catch the thief by lowering their standards to spot the tiny needle in the haystack, they end up stopping all 4,000 innocent tourists. The security system becomes useless because it's too noisy to work.

The Solution: A New Tool Called "CerberusAI"

To prove these ideas, the authors built a new open-source tool called CerberusAI (named after the three-headed dog guarding the underworld).

What it does: It's a simulation lab. It lets researchers set up a "fake" AI model and then send armies of "fake" attackers against it to see if the security holds up.
Why it matters: Before this, researchers mostly tested security by having one bad guy attack a model. CerberusAI lets them test what happens when an organized, coordinated army attacks.

The Big Takeaway

The paper concludes that we need to change our thinking.

Old Thinking: "Is this one specific person asking too many questions?"
New Thinking: "Is this group of people trying to steal my secret, even if they are hiding in a crowd?"

The authors argue that for military and critical infrastructure (like power grids or defense systems), we can no longer trust security systems that only look at individuals. We need smarter guards that can see the whole picture and understand the intent of the questions, not just how many questions are being asked.

In short: The paper proves that if you only guard against one thief at a time, a coordinated group of thieves will easily steal your secrets. We need a new kind of security that can spot the whole gang, even when they are hiding in plain sight.

Technical Summary: AI Model Extraction Attacks: Bypassing Single-Client Assumptions in Defenses

Problem Statement

The security of Artificial Intelligence (AI) models deployed in military Command and Control (C2) systems and critical infrastructure is increasingly threatened by Model Extraction Attacks (MEAs). In these attacks, adversaries systematically query a target model to replicate its decision boundary, enabling the theft of intellectual property and the offline development of adversarial attacks (e.g., evading Intrusion Detection Systems).

Current defense mechanisms, such as Protecting Against Deep Neural Network Model Stealing Attacks (PRADA), Query Unlearning (QUEEN), and Model-Guardian, rely heavily on the Single Client Assumption (SCA). This assumption posits that malicious activity originates from a single, isolated identity (e.g., one IP address or API key). Defenses operate by monitoring per-client request patterns (e.g., inter-arrival times) and triggering alarms when statistical deviations occur.

The paper argues that the SCA is fundamentally invalid in modern, networked operational environments, particularly against Advanced Persistent Threats (APTs). In these scenarios, adversaries possess the resources to distribute probing activities across many coordinated clients (e.g., botnets), thereby rendering per-client statistical defenses ineffective. Furthermore, even global aggregation approaches are vulnerable to adaptive traffic mixing, where attackers blend malicious probes with legitimate background traffic to mask statistical signatures.

Methodology

To systematically demonstrate the failure of the SCA and evaluate distributed attack scenarios, the authors developed CerberusAI, a modular, open-source framework for reproducible model-stealing research.

The CerberusAI Framework

CerberusAI is designed to simulate complex, distributed attack scenarios aligned with Software Defined Defence (SDD) architectures. Its core components include:

Declarative Configuration: Experiments are defined via YAML files, ensuring reproducibility by fixing parameters for target models, attack strategies, and defense hyperparameters.
Experiment Orchestrator: Uses a factory pattern to instantiate components, wrapping the target model in a DefendedModel proxy that routes requests through defense logic (e.g., PRADA) before reaching the model.
Distributed Attack Manager: Simulates botnet-like structures by generating $N$ $N$ virtual client identities. It implements specific scheduling strategies to bypass defenses:
1. Round-Robin Scheduling: Distributes the total query budget cyclically across $N$ clients. This ensures no single client exceeds the minimum sample size ( $N_{min}$ ) required for statistical significance in detection logic.
2. Adaptive Traffic Mixing: Injects a high ratio of benign traffic into the attack stream. This obfuscates the statistical distribution of the total traffic, making it indistinguishable from legitimate user behavior.

Attack Formalization

The authors formalize the distributed attacker as a set of clients $C = \{c_1, \dots, c_N\}$ that partition the necessary query set $Q$ into disjoint subsets $Q_{c_i}$ . If each subset $|Q_{c_i}|$ remains below the critical threshold $\epsilon$ (the minimum sample size for statistical tests), the defense mechanism $D$ , which evaluates clients in isolation, fails to trigger an alarm.

Key Contributions

Systematic Demonstration of SCA Vulnerability: The paper provides empirical evidence that coordinated, distributed query strategies can reduce the detection performance of established defenses (specifically PRADA) to zero.
Analysis of Global Defense Limits: The authors demonstrate that even naive global request aggregation fails when attackers employ adaptive traffic mixing. This results in a massive increase in False Positives (FP), rendering the defense operationally useless (effectively a denial of service for legitimate users).
Introduction of CerberusAI: The release of a modular, open-source framework that enables the simulation of distributed, adaptive MEA scenarios, addressing the lack of standardized evaluation tools in the field.

Experimental Results

The authors evaluated the PRADA defense mechanism using the MNIST dataset and a Convolutional Neural Network (CNN) target model. The experiments utilized the Momentum Iterative Fast Gradient Sign Method (MI-FGSM) for attack generation.

Scenario 1 (Baseline - Single Client): A single attacker against the standard PRADA defense. The system achieved an F1-score of 63.2% (Precision: 100%, Recall: 46.2%), confirming the defense functions as intended under the SCA.
Scenario 2 (Distributed Attack): The attack budget (2,500 queries) was distributed among 400 clients using round-robin scheduling. Each client sent only ~6-7 queries, well below the $N_{min}=50$ $N_{min} = 50$ threshold.
- Result: The F1-score dropped to 0.0%. The defense failed to detect the attack entirely because no single client triggered the statistical analysis.
Scenario 3 (Global Defense): A modified defense aggregated requests across all 400 clients.
- Result: The F1-score recovered to 90.8% (Precision: 100%, Recall: 83.1%), showing that global aggregation can detect simple distributed attacks.
Scenario 4 (Adaptive Traffic Mixing): The attacker mixed malicious queries with 99% benign traffic while using round-robin distribution.
- Result: The F1-score collapsed to 2.0%. While Recall remained high (99.3%), Precision plummeted to 1.0%. The system generated a flood of false alarms, making it operationally unusable.

Significance and Claims

The paper claims that the SCA represents a structural security gap in current AI defense doctrines. The findings indicate that:

SCA is Invalid: In the presence of coordinated, resource-rich adversaries (such as APTs), defenses relying on isolated client analysis are blind to distributed attacks.
Statistical Defenses are Fragile: Purely statistical approaches (based on inter-arrival times or distribution matching) can be arbitrarily manipulated through distribution and noise injection.
Paradigm Shift Required: Future research must move beyond client-centric anomaly detection. The authors argue for a shift toward stateful, identity-independent defense architectures that monitor the global state of the model and analyze the semantic content of queries (i.e., "What information content do these queries extract?") rather than focusing solely on their statistical distribution.

The work supports NATO's strategic goals by providing a methodology for "threat-informed evaluation" and highlighting the need for resilient AI security in federated and multi-domain operations. The authors conclude that protecting AI components in critical infrastructure requires moving away from assumptions of isolated attackers toward robust, learning-enabled security architectures capable of recognizing coordinated adversarial behavior.