Technical Summary: Why Geometric Continuity Emerges in Deep Neural Networks

Problem Statement

New observations suggest that weight matrices in adjacent layers of deep neural networks, particularly large language models (LLMs), exhibit geometric continuity: their leading singular vectors point in similar directions and form smooth trajectories across network depth. Although this phenomenon is empirically documented and linked to practical benefits such as layer pruning and partial parameter sharing across layers, its fundamental origin remains unclear. It is uncertain whether this continuity is an inevitable consequence of training deep networks or if it requires specific architectural prerequisites. This contribution investigates the mechanistic origins of geometric continuity to determine whether it is an artifact of specific optimizers or a structural byproduct of learning in residual networks.

Methodology

The authors employ a combination of controlled ablation studies on "toy" Multi-Layer Perceptrons (MLPs) and small Transformers, alongside an analysis of pre-trained LLMs.

1. Toy-MLP Experiments (MNIST):

   • Architecture: A residual MLP with 16 layers and 256 hidden dimensions.
   • Variables: The study systematically isolates architectural components by:
     • Removing residual connections (NoRes).
     • Removing activation functions (Res+None).
     • Replacing standard activations with rotation-equivariant nonlinearities (Res+Radial, using $\sigma_{rad}(x) = x \cdot \tanh(\|x\|)$).
     • Varying initialization scales and optimizer settings (e.g., Adam with $\beta_2=0$).
   • Metrics: The authors measure the cosine similarity of the leading singular vectors ($v_1$ for the input space, $u_1$ for the output space) between adjacent layers. They also define a $\sigma^2$-weighted alignment metric to evaluate continuity across the entire singular spectrum and track cumulative gradient coherence.

2. Transformer Experiments (WikiText-103 & Pre-trained LLMs):

   • Small Scale: A Llama-like Transformer (8 layers, 34 million parameters) trained on WikiText-103. The authors perform ablations on MLP activations to test their specific influence on various projection matrices (Q, K, V, Gate, Up, O, Down).
   • Large Scale: Analysis of pre-trained models such as Llama-3.1-8B and models with up to 70 billion parameters to verify whether the observed patterns hold at scale.

Main Contributions and Results

1. Two Necessary Conditions for Geometric Continuity

The contribution identifies two distinct mechanisms required for the emergence and persistence of geometric continuity:

• Condition 1: Residual Connections (Gradient Coherence):
  Residual connections ensure that adjacent layers receive similar error signals. In residual networks, the gradient with respect to a layer's input contains an identity term ($I$), generating coherent gradient propagation across layers. Empirical results show that without residual connections, gradient coherence drops to random values (~0.05) and no weight continuity develops.
• Condition 2: Symmetry-Breaking Nonlinearities (Stabilization of the Coordinate System):
  While residual connections provide coherent gradients, they are insufficient to maintain weight continuity without symmetry breaking.
  • The Failure of "Res+None": Removing the activation function (leaving a linear residual network) leads to high gradient coherence (0.93) but low weight continuity (0.22). The weights initially form a continuous structure but collapse due to rotational drift.
  • Mechanism: In linear networks, weight products possess a continuous rotationally symmetric property ($SO(d)$), creating flat directions in the loss landscape where weights can drift independently. Element-wise activations (ReLU, GELU, SiLU) break this symmetry, forcing layers into a shared coordinate frame and preventing drift.
  • Nonlinearity vs. Symmetry Breaking: Crucially, nonlinearity alone is not the active factor. A rotation-equivariant activation (Res+Radial), which is nonlinear but preserves $SO(d)$ symmetry, fails to maintain continuity (weight $v_1 \approx 0.22$). This isolates symmetry breaking as the critical factor.

2. Distinct Roles of Activation and Normalization

• Activation: Concentrates geometric continuity in the leading singular direction ($v_1$).
• LayerNorm: Breaks symmetry in a different manner by distributing continuity across multiple singular directions. In combination, the activation mode concentrated on $v_1$ dominates.

3. Projection-Specific Continuity in Transformers

In Transformer architectures, geometric continuity is not uniform but projection-specific, determined by the proximity of a nonlinearity and the direction of information flow relative to the residual stream:

• Continuity in the Input Space ($v_1$): Projections that read from the residual stream and are followed by a nonlinearity develop strong continuity in the input space. These include Q, K (stabilized by Softmax), Gate (stabilized by the MLP activation), and Up (stabilized by the gating multiplication with the activated Gate).
• Continuity in the Output Space ($u_1$): Projections that write into the residual stream and receive signals after the nonlinearity develop continuity in the output space. These include O and Down.
• Low Continuity: The V (Value) projection develops only low continuity in both spaces, as it lacks an adjacent nonlinearity to break symmetry and anchor its frame.

4. Dynamics of Gradient-to-Weight Transfer

The study reveals a causal sequence:

1. Gradient Coherence: A coherent gradient distribution across layers exists immediately at initialization (before weight updates) due to residual connections.
2. Weight Formation: As training progresses, weight updates accumulate. Once the magnitude of accumulated updates ($\|\Delta W\|$) dominates the initial weights ($\|W^{(0)}\|$), the weight structure aligns with the cumulative gradient structure.
3. Stabilization: Symmetry-breaking nonlinearities prevent the "rotational drift" that would otherwise decouple the weight structure from the coherent gradient signal.

Significance and Claims

The contribution claims that geometric continuity is a structural byproduct of learning in residual networks with symmetry-breaking nonlinearities, not an artifact of specific optimizers or training tricks.

• Mechanistic Explanation: The work provides a mechanistic explanation for why adjacent layers in LLMs share similar singular vector structures, linking architectural decisions (residuals, activations) directly to emergent weight geometry.
• Validation of Compression Strategies: The results offer a theoretical foundation for existing model compression techniques (e.g., layer pruning, partial parameter sharing across layers). The contribution argues that since continuity naturally emerges from the learning process, well-trained models inherently possess the cross-layer similarities that make these compression strategies feasible.
• Implications for Design: The projection-specific pattern suggests that compression or parameter-sharing strategies should respect the specific space (input vs. output) in which each projection is structured. For example, Q, K, Gate, and Up share bases in the input space, while O and Down share bases in the output space, whereas V offers no obvious opportunity for cross-layer sharing.

The authors remain modest regarding their claims, noting that while the mechanism for the MLP and the Gate projection (via direct ablation) is clear, the claims for Q/K and V in Transformers rely on indirect evidence or the absence of a nonlinearity. They also acknowledge that while the patterns hold across various scales (up to 70 billion parameters), the specific training dynamics of rotational drift at larger scales have not yet been fully verified.