💻 computer science

The Three-Force Phase Diagram of Mixture-of Experts: Gradient Alignment Theory and Causal Veri cation on Mixtral 8×7B

This paper introduces the cross-expert gradient alignment metric (Γ) to demonstrate that standard Mixture-of-Experts training, particularly in Mixtral 8×7B, is fundamentally driven by a structural zero-sum competition caused by the combined negative effects of softmax coupling and top-k normalization, which creates a persistent negative alignment attractor that can only be overcome by removing these specific architectural constraints.

Original authors: 庆君张

Published 2026-07-22

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: 庆君张

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Great Expert Showdown: Why AI Models Fight Themselves

Imagine you are building a massive library to teach a robot how to speak. Instead of hiring one giant, all-knowing librarian who tries to remember everything, you hire a team of eight specialized experts. Maybe one is great at math, another at poetry, and a third at history. This is the idea behind Mixture-of-Experts (MoE), a clever trick used in modern AI to make models huge and smart without making them too slow to run. The computer acts like a smart manager (called a "router") that looks at a question and decides which two experts should answer it.

But here is the tricky part: how do these experts learn? In a perfect world, they would work together like a well-oiled machine, each getting better at their own job without bothering the others. However, in the messy reality of training AI, the "manager" has to make choices. If the manager decides to give more attention to the math expert, it might have to take attention away from the history expert. This creates a tug-of-war. Scientists have long wondered: Is this team learning to cooperate, or are they stuck in a constant, zero-sum fight where one person's gain is another's loss? Understanding this is crucial because if the experts are fighting too hard, they might stop learning new things or even start acting exactly the same, ruining the model's intelligence.

The Paper's Discovery: A Three-Way Tug-of-War

In this research, a scientist named Zhang Qingjun decided to peek behind the curtain of a famous AI model called Mixtral 8×7B to see exactly how these experts are fighting. Instead of just guessing, they invented a new "thermometer" called Gamma (Γ). This tool measures the relationship between the experts' learning signals. If Gamma is positive, the experts are in a "turbulent" state where they are synergistically reinforcing each other, which paradoxically accelerates their collapse into identical behavior. If it's zero, they are ignoring each other and learning independently (a calm "laminar" state). If Gamma is negative, they are in a zero-sum battle, where helping one expert hurts the others.

The study found that the standard Mixtral model is stuck in a negative state (Γ = -0.107). This means that under normal settings, the model is fundamentally a zero-sum game. Every time the "manager" improves the routing for one expert, it accidentally makes it harder for the others to learn. The researcher ran seven different experiments to figure out why this happens and to see if they could fix it.

The Three Forces at Play
The paper breaks down this negative fighting into three specific forces, like a physics equation:

The Good Force (+0.030): Because all experts share the same basic brain structure, they naturally have a tiny bit of positive alignment. They want to help each other.
The Soft Force (-0.044): The "manager" uses a mathematical tool called softmax to decide who gets picked. This tool creates a subtle competition because the probabilities must add up to 100%.
The Heavy Force (-0.124): The biggest culprit is the rule that the model picks exactly 2 experts (top-2) and forces their weights to add up to 1. This creates a strict "zero-sum" constraint: if Expert A gets a bigger slice of the pie, Expert B must get a smaller slice. This is the main reason the experts are fighting.

When you add these up, the heavy fighting force wins, leaving the model with a net negative score of -0.107. The study proves that this isn't just a fluke of the specific data or weights; it's a structural rule of the system. Even when the researcher tried to mess with the model by adding noise, changing the learning tasks, or freezing parts of the brain, the fighting continued. The negative state is a "structural attractor"—a trap the model falls into by design.

The "Inverted-U" Surprise
One of the most playful findings is how the number of experts picked changes the fighting. The researcher tested picking 1, 2, 3, 4, 6, or 8 experts.

Picking 1 expert (k=1): The fighting stops completely! Gamma becomes 0.000. With only one expert, there is no one to fight with, so the model enters a perfect "laminar" (calm) state.
Picking 2 experts (k=2): The fighting is at its peak (most negative). This is the native Mixtral setting.
Picking more experts (k=8): The fighting gets weaker again (Gamma rises to -0.045). Why? Because the "pie" is being split among so many people that the competition between any two specific experts becomes diluted.

This creates an inverted-U shape: the competition is weakest when you pick just one, gets strongest when you pick two, and then fades as you pick more.

Can We Fix It?
The paper suggests two ways to escape the fighting trap:

Pick only one expert (k=1): This eliminates the competition entirely, creating a calm learning environment. However, the paper notes this comes with a trade-off: it may degrade overall model quality because fewer experts are active per token, even though the learning process itself is cleaner.
Use "Hard Routing": Instead of letting the manager learn to choose experts, you can assign experts to specific topics permanently (like a fixed schedule). The study tested this on a smaller vision model and found it reduced the fighting to almost zero (Gamma = -0.009), creating a near-perfect calm state.

What This Means
The paper concludes with a "Competitive Routing Lemma": if you build a system that picks 2 or more experts and forces their choices to add up to a fixed total, you are mathematically guaranteed to create a zero-sum fight. The authors didn't just guess this; they measured it across 32 layers of a massive model and proved it holds true. While the model still learns (it's not a failure), this internal fighting makes it harder for the experts to specialize and can cause problems when the model tries to learn new tasks later on. The solution, the paper suggests, is to redesign how the "manager" picks experts to avoid this built-in conflict.

Technical Summary: The Three-Force Phase Diagram of Mixture-of-Experts

1. Problem Statement

Mixture-of-Experts (MoE) architectures scale model capacity by activating a subset of sub-networks per token, yet they suffer from "expert collapse," where experts converge to functionally identical representations. While existing literature addresses surface-level symptoms like load imbalance through auxiliary losses or capacity constraints, the underlying gradient dynamics driving expert alignment remain undiagnosed. Specifically, it is unclear whether the coupling between experts in production-scale language models (e.g., Mixtral 8×7B) is synergistic, independent, or competitive, and what structural forces govern this coupling.

2. Methodology

The authors introduce $\Gamma$ (Gamma), a diagnostic metric defined as the mean pairwise cosine similarity of router-logit gradients across all expert pairs. This metric quantifies inter-expert coupling:

$\Gamma > 0$ (Turbulent): Synergistic reinforcement.
$\Gamma \approx 0$ (Laminar): Independent learning.
$\Gamma < 0$ (Negative-aligned): Zero-sum competition.

The study employs a systematic causal diagnosis on Mixtral 8×7B (47B parameters, 32 layers, softmax top-2 routing) using seven distinct experiments via monkey-patching and gradient hooks, without modifying model weights:

Baseline Measurement: Establishing native $\Gamma$ .
Perturbation Tests: Applying LoRA continual learning, gate unfreezing, and noise sweeps (0–100% random tokens) to test if $\Gamma$ is weight-dependent.
2×2 Factorial Decomposition: Isolating the effects of activation functions (softmax vs. sigmoid) and top-k weight normalization (on vs. off).
Top-k Ablation: Varying $k$ from 1 to 8 to observe competition intensity.
Cross-Architecture Validation: Testing a vision-based "Tile MosaicModel" with softmax and hard routing to verify universality.

3. Key Results

A. The Structural Nature of Competition

Native State: The baseline Mixtral 8×7B exhibits a uniform $\Gamma = -0.107$ across all 32 layers.
Structural Attractor: Perturbation experiments (LoRA, gate unfreezing, noise injection) failed to shift $\Gamma$ across zero. Even with 100% random input tokens, $\Gamma$ only shifted slightly (+0.015). This confirms that the negative-aligned state is a structural attractor of the top-2 softmax routing mechanism, independent of specific weight values or data distributions.

B. The Three-Force Decomposition

A 2×2 factorial analysis decomposed the net $\Gamma$ into three opposing forces:

Architectural Positive Alignment (+0.030): Arising from the shared FFN backbone, where all experts process similar hidden states.
Softmax Denominator Coupling (−0.044): Caused by the derivative $\partial\sigma_i/\partial z_j = -\sigma_i\sigma_j$ , creating global negative coupling.
Top-k Zero-Sum Normalization (−0.124): The dominant force, arising from the constraint $\sum w_i = 1$ , which implies $\partial w_i/\partial w_j = -1$ .

The net effect is: $+0.030 - 0.044 - 0.124 = -0.107$ . Crucially, only when both softmax coupling and top-k normalization are removed (Condition C) does $\Gamma$ cross into positive territory (+0.030).

C. The Inverted-U Relationship

Top-k ablation revealed an inverted-U relationship between the number of active experts ( $k$ ) and competition intensity:

$k=1$ : $\Gamma = 0.000$ (Perfect Laminar). No competition pool exists.
$k=2$ (Native): $\Gamma = -0.107$ (Maximum Competition). The zero-sum constraint is most concentrated between two experts.
$k \to 8$ : $\Gamma$ weakens (e.g., $-0.045$ at $k=8$ ) as the zero-sum constraint dilutes across a larger pool, though it remains negative.

D. Cross-Architecture Validation

The "Tile MosaicModel" (vision task) confirmed the universality of these dynamics:

Softmax routing (no top-k): $\Gamma = -0.031$ , matching the magnitude of the softmax coupling term in Mixtral.
Hard routing (fixed weights): $\Gamma \approx -0.009$ , confirming that eliminating learned routing competition restores the laminar regime.

4. Key Contributions

The $\Gamma$ Diagnostic: A gradient-level metric applicable to any MoE architecture without weight modification, quantifying inter-expert coupling strength.
The Three-Force Model: A causal decomposition of MoE gradient alignment into architectural positive alignment, softmax denominator coupling, and top-k zero-sum normalization.
The Competitive Routing Lemma: A formal result stating that any routing mechanism selecting $k \ge 2$ experts with normalized weights necessarily produces $\Gamma < 0$ , with intensity peaking at $k=2$ .
Unified Phase Diagram: A spectrum demonstrating that these dynamics hold across different scales (36M to 47B), modalities (vision vs. language), and architectures (Transformer vs. WRN).
Design Implications: Identification of $k=1$ routing and hard/fixed routing as sufficient conditions for achieving the laminar regime ( $\Gamma \approx 0$ ).

5. Significance and Claims

The paper claims to provide a unified gradient-alignment explanation for MoE training dynamics. It challenges the notion that expert collapse is solely a symptom of load imbalance, reframing it as a consequence of the zero-sum nature of normalized routing gradients.

Clarification on Training Failure: The authors explicitly state that $\Gamma < 0$ does not imply training failure; Mixtral trains successfully because the competition is weak ( $|\Gamma| \ll 1$ ) and affects routing weights rather than directly locking expert outputs.
Implications for Continual Learning: The paper suggests that the zero-sum competition becomes pathological in continual learning settings, where improving routing for new tasks inherently degrades routing for old tasks, amplifying catastrophic forgetting.
Modest Scope: The authors acknowledge that the causal chain is primarily established on Mixtral 8×7B. Cross-model validation on Qwen1.5-MoE and DeepSeek-V2-Lite is noted as ongoing work. The study focuses on static measurements with frozen weights, leaving dynamic tracking during full optimization as a future direction.

In conclusion, the paper establishes that the "zero-sum trap" in MoE is a structural inevitability of $k \ge 2$ normalized routing, and that escaping this regime requires architectural interventions that remove the competition pool ( $k=1$ ) or the learned competition itself (hard routing).