Safety-Oriented Routing Analysis of Mixtral MoE Under Benign and Harmful Prompts

Technical Summary: Safety-Oriented Routing Analysis of Mixtral MoE Under Benign and Harmful Prompts

Problem Statement
Sparse Mixture-of-Experts (MoE) language models, such as Mixtral 8x7B, activate only a small subset of parameters for each token, relying on a router to determine expert engagement. While prior work suggests routing patterns in Mixtral align more with syntax than semantic domain, it remains unclear whether router behavior shifts systematically when processing benign versus harmful prompts. This paper investigates whether safety-relevant behavior in Mixtral is associated with identifiable internal routing mechanisms, specifically examining if harmful prompts induce distinct expert selection frequencies or gradient sensitivities compared to benign prompts.

Methodology
The authors developed a unified analysis framework to study Mixtral 8x7B-Instruct using two complementary signals:

1. Activation-Based Scores: Derived from the frequency of expert selection by the router gate. This captures how often specific experts are engaged across tokens.
2. Gradient-Based Scores: Derived from the absolute gradient magnitude of the router gate weights with respect to the sequence loss. This captures how influential specific router parameters are for the model's loss.

The study utilized 20 prompts (10 benign, 10 harmful) for descriptive analysis and 100 harmful prompts for intervention experiments. The analysis proceeded at three levels:

• Expert-Level: Analyzing the distribution of scores across all 256 layer-expert pairs (32 layers × 8 experts) using metrics like $k_{80}$, $k_{90}$, and $k_{95}$ (number of pairs needed to cover 80–95% of score mass) and elbow cutoffs.
• Layer-Level: Computing concentration metrics (dominant score, top-2 sum, entropy, effective expert count) for each of the 32 layers.
• Expert Classification: Categorizing layer-expert pairs as "benign-dominant," "malicious-dominant," or "shared" based on the gap between group averages and minimum magnitude thresholds.
• Intervention: Suppressing the top five benign-dominant experts identified by each method during generation on 100 harmful prompts to measure changes in refusal behavior (restricted vs. non-restricted responses).

Key Contributions

1. Unified Framework: Introduction of a dual-signal framework (activation and gradient) for analyzing internal routing behavior in Mixtral under safety-relevant conditions.
2. Signal Divergence: Demonstration that activation-based routing is broad and long-tailed, whereas gradient-based importance is substantially more concentrated.
3. Layer-Specific Selectivity: Identification that activation-based selectivity peaks in middle-to-late layers (8–15), while gradient-based importance concentrates sharply in the final layers.
4. Expert Overlap: Classification results showing that the majority of experts are shared between benign and harmful groups, with only a limited subset showing clear group preference.
5. Causal Evidence: Preliminary intervention results showing that suppressing identified experts reduces refusal rates, providing mechanistic links between routing behavior and safety outputs.

Results

• Expert-Level Distribution: Activation scores are distributed across a large portion of the model (requiring ~150+ pairs to cover 80% of mass), while gradient scores are highly concentrated (requiring ~50–60 pairs). Despite these distributional differences, the overall routing distributions for benign and harmful groups remain close, with only modest separation.
• Layer-Level Dynamics:
  • Activation: The most selective region is layers 8–15. Harmful prompts become slightly more diffuse in the final layers (24–31).
  • Gradient: Importance rises sharply toward the end of the network, with the strongest concentration at the final layer (layer 31). Effective expert counts drop significantly in these final layers for both groups.
• Classification and Overlap:
  • Activation classification identified 22 benign-dominant and 18 malicious-dominant experts out of 256.
  • Gradient classification identified only 6 benign-dominant and 5 malicious-dominant experts.
  • Crucially, the top-10 ranked experts by gradient score showed strong overlap between benign and harmful groups (8/10 shared), whereas activation top-10s showed limited overlap (3/10 shared). This suggests gradient importance concentrates both prompt types on a common late-layer core.
• Intervention Outcomes:
  • Activation Suppression: Reduced restricted responses from 24 to 14 (41.7% reduction) but caused 8 unintended reversals (non-restricted to restricted).
  • Gradient Suppression: Reduced restricted responses from 34 to 22 (35.3% reduction) with fewer unintended reversals (3).
  • Neither intervention caused a universal behavioral collapse, indicating that safety behavior is not controlled by a single fixed subset of experts.

Significance and Claims
The paper claims that safety-relevant routing in Mixtral is subtle, depth-dependent, and distributed rather than dominated by a fixed set of "harmful" experts. The findings suggest that while harmful prompts do shift internal routing behavior, these shifts are not captured by a single metric. Instead, activation signals reveal broad routing preferences, while gradient signals highlight loss-sensitive concentration in late layers.

The authors conclude that relying on a single internal metric provides an incomplete picture of MoE safety. A more informative approach involves jointly analyzing routing frequency, loss-sensitive importance, and intervention behavior. The study provides preliminary causal evidence that identified experts contribute to refusal behavior but emphasizes that the mechanism is complex and distributed across the network depth.