🤖 machine learning

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

This paper introduces RepetitionCurse, a low-cost black-box attack that exploits the lack of load balancing constraints in Mixture-of-Experts LLMs by using simple repetitive token patterns to force severe routing concentration, thereby creating computational bottlenecks and significantly degrading inference latency and service availability.

Original authors: Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ruixuan Huang, Qingyue Wang, Hantao Huang, Yudong Gao, Dong Chen, Shuai Wang, Wei Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, high-tech library where thousands of books (the "experts") are stored on different shelves (the "GPUs"). When a librarian (the "router") gets a request, they don't just grab one book; they pick the top few best books to answer the question. To make things fast, the library splits the work: some shelves handle the first few books, others handle the next batch, and they all work at the same time. This is how modern AI models called Mixture-of-Experts (MoE) work. They are designed to be super efficient by spreading the work out evenly.

However, a new paper called "RepetitionCurse" reveals a sneaky flaw in how this library operates.

The Problem: The "Copy-Paste" Trick

The researchers discovered that if you ask the librarian a question using a very strange, repetitive pattern (like typing "AI AI AI AI..." over and over), the librarian gets confused.

Normally, the librarian looks at the meaning of your words to decide which books to pick. But when you use a repetitive pattern, the librarian stops looking for meaning and starts acting like a broken record. Instead of spreading the work out across all the shelves, the librarian suddenly decides that only one specific shelf has the right books for this repetitive pattern.

The Result: A Traffic Jam

Here is what happens next:

The Bottleneck: All the work gets dumped onto that single shelf. The librarians on that shelf are running around frantically, trying to find all the books.
The Idle Workers: Meanwhile, the librarians on all the other shelves are standing around doing nothing, waiting for the busy shelf to finish.
The Delay: Because the whole system has to wait for that one slow shelf to catch up before it can move on to the next step, the entire library grinds to a halt.

In the world of AI, this delay is called Time-to-First-Token (TTFT). It's the time it takes for the AI to say its first word. Under normal conditions, this is fast. Under this "RepetitionCurse" attack, the AI might take 2 to 3 times longer to start speaking.

Why This Matters

The paper calls this a Denial-of-Service (DoS) attack. It's like someone walking into a fast-food restaurant and ordering 1,000 identical, complicated burgers at once. The kitchen gets clogged trying to make those specific burgers, and suddenly, the person who just wanted a simple coffee has to wait an hour.

The Attack is Simple: You don't need to be a hacker genius. You just need to type repetitive words. The paper calls this a "black-box" attack, meaning you don't need to know how the AI's brain works inside; you just need to know that repeating words breaks the balance.
It Works Everywhere: The researchers tested this on 139 different AI models (including popular ones like Mixtral and Qwen). They found that almost all of them are vulnerable.
The More You Scale, The Worse It Gets: Ironically, the more powerful the AI system is (using more computers to work in parallel), the easier it is to break. The researchers found that using more computers to split the work actually makes the traffic jam worse when this attack is used.

The Bottom Line

The paper concludes that while splitting the work among many computers makes AI faster, it also creates a weak spot. If someone uses a simple, repetitive trick, they can force the system to ignore its own efficiency rules, causing it to slow down dramatically and potentially break its promises to users (like "we will reply within 2 seconds").

The authors suggest that until AI companies figure out a better way to balance the work automatically, they might need to be careful about how many computers they use at once, or they risk their systems being held hostage by a simple "AI AI AI" loop.

Technical Summary: RepetitionCurse

Problem Statement

The paper identifies a critical system-level vulnerability in Mixture-of-Experts (MoE) Large Language Models (LLMs) deployed with Expert Parallelism (EP). While MoE architectures are designed to scale efficiently by dynamically routing tokens to a subset of experts, the absence of explicit load-balancing constraints during inference creates a susceptibility to Denial-of-Service (DoS) attacks.

Specifically, the authors demonstrate that out-of-distribution (OOD) inputs can manipulate the router mechanism to concentrate token routing onto a small, shared subset of top- $k$ experts. In an EP deployment where distinct experts are hosted on separate GPUs, this concentration causes severe load imbalance: a few GPUs become computational bottlenecks (stragglers) while others remain idle, waiting for synchronization (all-reduce). This converts an efficiency mechanism into an attack vector that inflates Time-to-First-Token (TTFT), violating Service-Level Agreements (SLAs) and degrading service quality for all users in a batch.

Methodology: RepetitionCurse

The authors propose RepetitionCurse, a black-box attack strategy that exploits the discrepancy between training-time balancing and inference-time routing.

Core Mechanism: The attack constructs prompts using simple, repetitive token patterns (e.g., "AIAI...").
Theoretical Basis: The router's load balancing relies on the assumption that hidden states are semantically diverse. By repeating tokens, the attacker minimizes the variance in layer-wise hidden representations, collapsing them into a near-degenerate cluster. This forces the router to deterministically select the same top- $k$ experts for the entire sequence.
Black-Box Nature: The method requires no access to model parameters, gradients, or the specific Expert-GPU mapping. It operates purely on input prompts.
Attack Intensity: The attack is parameterized by $\alpha$ , the fraction of attack samples within a batch. The study evaluates scenarios ranging from mixed-user batches ( $0 < \alpha < 1$ ) to attack-only batches ( $\alpha = 1$ ).

Key Contributions

Vulnerability Identification: The paper is the first to demonstrate that routing imbalance in MoE models can be weaponized to launch DoS attacks, shifting the focus from resource exhaustion via long outputs to resource underutilization via routing concentration.
Attack Proposal: Introduction of RepetitionCurse, a model-agnostic, gradient-free method that amplifies prefilling latency and degrades TTFT using repetitive tokens.
Comprehensive Measurement: A large-scale empirical study across 139 MoE configurations and 13 popular models (including Mixtral, Qwen3, DeepSeek, and Llama variants) to quantify the severity of latency amplification and identify system-level factors.

Experimental Results

The authors evaluated the attack on 8-GPU clusters (and scaled to 16/32 GPUs for specific models) using models like Mixtral-8x7B and Qwen3-30B-A3B.

Universality: The vulnerability is intrinsic and universal. Nearly any token in the vocabulary can serve as a trigger for routing concentration. The "bottleneck coverage" (the probability a random token induces maximal concentration) approaches 1.0 for most models when the EP size matches the number of experts.
Latency Amplification:
- Under common 8-GPU EP settings, TTFT increased by 20% to 148% (e.g., up to 2.48 $\times$ for Mixtral-8x7B).
- For sparser models like Qwen3-30B-A3B with EP size = 32, TTFT increased by up to 115%.
- The attack significantly degrades SLA guarantees; for instance, P99 SLA compliance dropped from 99% to between 86.4% and 98.6% under attack.
Impact of EP Size: Higher degrees of Expert Parallelism substantially increase susceptibility. The attack surface expands as the EP size grows, creating a deployment dilemma where efficiency gains amplify security risks.
Defense Limitations: The authors investigated "vulnerability-aware" Expert-GPU mapping (placing vulnerable experts on different GPUs). While this offered conditional mitigation for models with moderate expert counts, it collapsed for highly sparse models or very high EP sizes, confirming the vulnerability persists regardless of mapping strategies in black-box scenarios.
Robustness: The attack is robust to prompt length (100 to 16k tokens) and context variations, indicating the vulnerability is systemic to the MoE architecture's handling of repetitive embeddings.

Significance and Claims

The paper claims to uncover a fundamental trade-off between parallelization efficiency and system robustness in MoE architectures. As the industry increasingly adopts highly sparse MoE models for cost-effective scaling, the reliance on dynamic routing without inference-time balancing creates a systemic weakness.

The authors position RepetitionCurse primarily as a diagnostic tool for researchers and system providers to evaluate the robustness of their serving infrastructure. They emphasize that current deployment strategies (specifically large EP sizes) inadvertently turn the system's parallelism against itself. The findings serve as a timely warning for cloud service providers to reconsider EP sizing and develop better inference-time load-balancing strategies to prevent SLA violations and service degradation.