💻 computer science

Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE

This paper identifies a novel supply-chain backdoor attack on Vision Mixture-of-Experts (MoE) models that exploits batch-dependent capacity overflow to remain dormant during small-batch security audits while activating high-success-rate attacks during large-scale deployment, effectively evading existing detection methods.

Original authors: Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of artificial intelligence, computers learn to see by processing images through vast digital networks. These networks are often built like a factory floor, where different specialized units, called "experts," handle specific parts of a task. To make these systems fast and efficient, engineers use a method where the computer only sends each piece of information to the expert best suited for it, leaving the others idle. This approach, known as a Mixture of Experts, allows models to become incredibly large and capable without slowing down. However, this efficiency relies on a strict rule: each expert can only handle a certain number of items at once. If too many items arrive at the same time, the system must drop some of them to keep running smoothly. This rule is designed to prevent the computer from crashing, but it creates a hidden vulnerability that security researchers have only just begun to understand.

A team of researchers has discovered that this very rule for managing workload can be turned into a secret switch for a malicious attack. They demonstrated that a computer model used for vision tasks, such as identifying traffic signs for self-driving cars, can be poisoned with a hidden trap. This trap remains completely invisible when the model is tested under normal, low-pressure conditions. Yet, the moment the model is put to work in a busy real-world scenario where it must process many images at once, the trap springs open. The researchers call this the "Capacity Overflow" attack. It is a stealthy form of sabotage that exploits the way these models handle stress, remaining dormant during security checks and activating only when the system is under heavy load.

The attack works by taking advantage of the model's internal traffic control. In a standard setup, when a batch of images is sent to the model, a router decides which expert handles each image. To ensure the system does not get overwhelmed, there is a limit on how many images each expert can process. If a batch is small, every image gets processed, and the model behaves normally. But if the batch is large, the experts reach their limit, and the system is forced to discard the extra images, sending them directly to the final output without further analysis. The researchers found that they could train a model to use this discarding mechanism as a trigger. They injected a hidden instruction into the early layers of the network that would cause the model to make a mistake, such as misidentifying a stop sign as a speed limit sign. To hide this instruction, they added a second layer of defense within the model itself, a "neutralizer" that cancels out the mistake whenever the system is running normally.

The brilliance of the attack lies in how it manipulates the conditions under which this neutralizer works. The neutralizer is trained to function only when every single image in a batch is processed. If the batch is small, the neutralizer sees all the images, does its job, and the model remains safe. However, if the batch is large, the system hits its capacity limit and begins dropping images. Because the neutralizer was not trained to handle missing information, it fails to cancel out the hidden instruction. The result is that the model suddenly starts making the malicious mistake, but only when it is processing a large number of images at once. This creates a scenario where a security auditor, testing the model with a small batch of images, sees a perfectly safe system, while a self-driving car in a busy city, processing a continuous stream of traffic, is tricked into dangerous behavior.

To prove this was possible, the researchers tested their method on two different types of vision models using standard image datasets. They successfully created models that could be triggered to misclassify images with a success rate between 76% and 87% when the batch size was large. In contrast, when the same models were tested with small batches, the attack failed almost entirely, with a success rate below 9%. The models also maintained their ability to correctly identify normal images, meaning the attack did not degrade the system's overall performance in a way that would give it away. The researchers showed that this attack could bypass several existing security tools designed to find hidden traps, because those tools typically only look at the model under low-load conditions where the trap is inactive.

This discovery reveals a fundamental blind spot in how we secure artificial intelligence. Current safety checks often assume that a model behaves the same way whether it is processing one image or a thousand. The researchers found that this assumption is false for these specific types of models. The behavior of the system changes depending on the load, and this change can be weaponized. The attack does not require any special hardware or complex external signals; it simply relies on the natural increase in traffic that happens when a system is deployed in the real world. By configuring the model to reduce its capacity limits during high-speed operation, an attacker can ensure that the neutralizer fails and the malicious behavior takes over.

The implications of this finding are significant for any industry that relies on large-scale vision models, particularly autonomous driving. A self-driving car might pass every safety inspection in a lab, where it is tested with a few images at a time, yet fail catastrophically on a highway where it must process a rapid stream of visual data. The researchers suggest that security evaluations need to change to reflect these real-world conditions. Audits must test models under heavy loads and varying batch sizes to ensure that the system remains safe when it is actually being used. Without these changes, the very mechanisms designed to make these models efficient and scalable could become the weakest link in their security, allowing hidden threats to remain undetected until it is too late.

Technical Summary: Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE

Problem Statement

Vision Mixture-of-Experts (MoE) architectures have become a standard paradigm for scaling Vision Transformers (ViTs) efficiently by utilizing sparse computation. To ensure computational stability and prevent expert overload, these models employ a capacity-bounded token dispatch mechanism. This mechanism limits the number of tokens each expert can process based on a predefined capacity factor ($cf$) and the inference batch size ( $N$ ). When the number of tokens assigned to an expert exceeds its capacity limit, the excess tokens are deterministically dropped and bypass the expert computation via residual connections.

The authors identify this batch-dependent behavior as a previously overlooked attack surface. While existing backdoor defenses typically operate under small-batch settings to facilitate detailed analysis (e.g., trigger reconstruction or per-sample inspection), they implicitly assume that backdoor behaviors are static and independent of inference load. This paper argues that the dynamic nature of token dropping in Vision MoE creates a "systematic blind spot" where malicious behaviors can remain dormant during standard security audits but activate under high-concurrency, large-batch deployment scenarios.

Methodology: The Capacity Overflow Attack

The proposed attack, Capacity Overflow, is a stealthy supply-chain backdoor that exploits the resource contention inherent in MoE token dispatch. It operates through a three-phase framework designed to create a conditional backdoor that activates solely based on inference parallelism (batch size) without requiring external triggers or hardware manipulation.

1. Implantation Phase

The adversary poisons a small fraction of the training data (20–25%) with a fixed visual trigger (e.g., a patch) mapped to a target class.

Location: The backdoor is implanted in an early MoE layer.
Mechanism: Only the parameters of the backdoor MoE layer and its immediately preceding attention block are updated. The router and subsequent layers remain frozen.
Objective: To establish a strong association between the trigger pattern and the target class in the intermediate representations.

2. Concealment Phase

To ensure the model behaves benignly under standard workloads (small batches), a neutralizer is trained in a deeper MoE layer.

Mechanism: The neutralizer is trained via KL distillation against a clean reference model to suppress the backdoor signal.
Critical Constraint: The neutralizer's router is frozen during training, and the training is conducted at a nominal capacity factor ( $cf \approx 1.05$ ). This ensures the neutralizer learns to suppress the backdoor only when all routed tokens are processed.
Fragility: Because the router is frozen, the neutralizer cannot adapt to token dropping. If tokens are dropped due to capacity overflow, the neutralizer receives incomplete information, causing the suppression chain to break.

3. Activation Phase

No additional training is required. The adversary configures the deployment environment to exploit the neutralizer's fragility.

Batch-Adaptive Capacity Factor: The attacker (or the model provider) sets a configuration where the capacity factor ($cf$) is high for small batches (preserving quality) but reduced for large batches (to improve throughput).
Ascending Dispatch Ordering: Tokens are dispatched in ascending order of priority. When overflow occurs in large batches, the most informative tokens (those critical for the neutralizer's function) are dropped first.
Result:
- Dormant Mode (Small Batch): High $cf$ ensures no overflow; the neutralizer functions correctly, suppressing the backdoor.
- Activation Mode (Large Batch): Low $cf$ induces token overflow; the neutralizer is bypassed, allowing the backdoor to trigger the malicious output.

Key Contributions

Novel Attack Vector: The introduction of Capacity Overflow, a conditional backdoor against Vision MoE models activated by inference-time resource contention (token overflow) rather than external signals.
Three-Phase Framework: A methodology that jointly implants a backdoor and a capacity-dependent neutralizer, achieving a sharp behavioral transition between benign and malicious states based on batch size.
Comprehensive Evaluation: Experiments on V-MoE and Swin-MoE across ImageNet-100 and GTSRB datasets.
- Attack Success Rate (ASR): Achieved 76–87% in activation mode (large batches) while maintaining dormant ASR below 9%.
- Clean Accuracy: The attack caused a minimal drop in clean accuracy (≤1.3 percentage points) compared to unattacked baselines.
Defense Evasion: The attack successfully evades state-of-the-art defenses including Neural Cleanse, STRIP, Fine-Pruning, and Activation Clustering. These defenses fail because they operate in the "dormant mode" (small batches) where the neutralizer effectively masks the backdoor.

Results and Performance

The experiments demonstrate that the attack is highly effective and stealthy:

Transition Threshold: The attack success rate increases monotonically as the capacity factor decreases. A sharp transition occurs between $cf \approx 0.8$ (dormant) and $cf \approx 0.3$ (activation).
Architecture Specifics: V-MoE showed higher activation ASR (up to 87.0%) compared to Swin-MoE (76.0–83.5%), attributed to Swin's windowed attention providing local redundancy that partially compensates for dropped tokens.
Defense Robustness: In defense evaluations, the anomaly indices for Neural Cleanse remained below detection thresholds (or resulted in false positives on non-target classes), and activation clustering failed to separate poisoned samples from clean ones due to the neutralizer's alignment of representations.

Significance and Implications

The paper claims that Capacity Overflow reveals a fundamental security risk arising from the batch-dependent execution in scalable Vision MoE architectures.

Systematic Blind Spot: The authors argue that current security evaluation practices, which rely on small-batch inspections, provide a "false sense of security" for MoE models. The vulnerability is not a flaw in a specific detector but a gap in the threat model that fails to consider load-dependent behavior.
Practicality: The attack leverages standard deployment configurations (adjusting capacity factors for throughput) and does not require specialized hardware or external triggers, making it highly practical for supply-chain attacks.
Call to Action: The findings necessitate a shift in security auditing for MoE models toward workload-aware audits that explicitly test for token dropping and capacity overflow regimes to avoid false assurances.

The paper concludes that while the attack is currently demonstrated on discriminative Vision MoE classifiers, the underlying vulnerability applies to any MoE implementation utilizing capacity-bounded token dropping, highlighting a critical area for future research in model security.