Technical Summary: Deciphering Shortcut Learning from an Evolutionary Game Theory Perspective

Problem Statement

Deep learning models frequently achieve high performance on benchmarks by relying on "shortcuts"—spurious correlations or non-essential features (e.g., background textures, dataset-specific artifacts) rather than core, task-relevant features. While the existence of shortcut learning is well-documented, the theoretical mechanisms governing its formation and evolution during training remain poorly understood. Existing literature often lacks precise definitions distinguishing core features from shortcuts and fails to explain how optimization dynamics (such as batch size and noise) influence the dominance of one feature type over another. Specifically, it is unclear why models initially favor easy-to-learn shortcuts and how stochasticity in data and optimization affects the long-term trajectory of feature learning.

Methodology

The authors propose a formal framework combining Evolutionary Game Theory (EGT) with Neural Tangent Kernel (NTK) analysis to model the dynamics of shortcut learning.

1. Formal Definitions

The paper establishes rigorous definitions for feature types in binary classification:

• Core Feature ($V^c$): A feature that is strictly correct and task-relevant. If fully learned, it guarantees generalization.
• Noise Feature: A feature with zero covariance with the label.
• Shortcut Feature ($V^{sc}$): A feature that is easy to learn but ultimately misleading. It is defined relative to a core feature: $V^{sc}$ is a shortcut if the model exhibits a stronger erroneous reliance on it compared to the core feature when evaluated on conflicting samples (samples where the shortcut and core features predict different labels).

2. Evolutionary Game-Theoretic Framework

The training process is modeled as an evolutionary game where:

• Players: Individual data samples.
• Strategies: The neural tangent feature direction ($\nabla_\theta f(X; \theta)$) adopted by a sample, categorized as either a Core Strategy (optimizing the core subnetwork) or a Shortcut Strategy (optimizing the shortcut subnetwork).
• Payoff: Defined as the reduction in loss for a sample when included in a mini-batch. This payoff depends on the alignment between the sample's tangent feature and the aggregate gradient update direction.
• Dynamics: The proportion of samples adopting each strategy evolves based on payoff differences. The authors model this using a discrete-time replicator dynamic with mutation (representing optimization noise) and analyze the limit as mutation rates approach zero to identify Stochastically Stable States (SSS).

3. Subnetwork Hypothesis and Assumptions

The authors assume the existence of decoupled subnetworks within the neural network:

• Core Subnetwork: Edges and neurons dedicated to modeling the core feature.
• Shortcut Subnetwork: Edges and neurons dedicated to modeling the shortcut feature.
• Decoupling: These subnetworks are structurally distinct, though they may share some edges (noise modeling).
• Initialization: Early training is dominated by features with high explained variance (often shortcuts), as shown via NTK analysis.

4. Stochastic Differential Equations (SDE)

To analyze the continuous-time evolution of training intensities ($w_1$ for core, $w_2$ for shortcut) and the proportion of core-strategy samples ($\alpha$), the authors derive a system of SDEs. This allows them to explicitly model the effects of:

• Data Noise ($\tau$): Interference from noise features that mimics signal.
• Optimization Noise ($\sigma$): Stochasticity introduced by mini-batch sampling.

Key Contributions and Results

1. Theoretical Characterization of Shortcut Bias

• Uniqueness of Core vs. Ubiquity of Shortcuts: The paper proves that core features are unique (Theorem 4), whereas shortcut features are ubiquitous whenever generalization accuracy is imperfect (Theorem 5).
• Initial Advantage of Shortcuts: Using NTK theory, the authors show that features with higher explained variance (often shortcuts like color or texture) dominate the principal eigenvectors of the kernel matrix, granting them an initial advantage in training (Theorem 6, 7).

2. Divergent Outcomes of GD vs. SGD

The central theoretical finding concerns the impact of optimization noise on the stochastically stable state of the system:

• Full-Batch Gradient Descent (GD): Corresponds to a deterministic dynamic with no optimization noise. The analysis (Theorem 11) shows that GD converges to a state where the Shortcut Strategy is stochastically stable. The model becomes trapped in the shortcut subnetwork.
• Mini-Batch Stochastic Gradient Descent (SGD): Introduces optimization noise via random batching. The analysis (Theorem 12) demonstrates that under specific conditions (sufficiently large $N$ and appropriate batch sizes), SGD shifts the system to a state where the Core Strategy becomes stochastically stable. The noise prevents the system from settling into the shortcut-dominated basin.

3. Role of Noise in Bias Formation

Through SDE analysis (Theorem 13), the paper reveals the distinct roles of noise sources:

• Data Noise ($\tau$): High levels of data noise (or noise that mimics signal) promote the formation of shortcut bias. It amplifies the training intensity of the shortcut subnetwork relative to the core.
• Optimization Noise ($\sigma$): High levels of optimization noise (e.g., smaller batch sizes) suppress shortcut bias. It facilitates the transition from shortcut dominance to core dominance by allowing the system to escape local minima associated with shortcut learning.

4. Empirical Verification

The authors validate their theory using the Colored-MNIST dataset (where color is a spurious shortcut and digit shape is the core feature):

• Initialization: PCA analysis confirms that color features account for a larger proportion of data variance than digit features, explaining the initial rapid learning of the shortcut.
• GD vs. SGD: Experiments show that full-batch GD leads to a model that separates samples by color (shortcut) but confuses digits, whereas mini-batch SGD leads to separation by digit (core) while confusing color.
• Subnetwork Identification: Fine-tuning experiments with L1 regularization reveal that distinct neurons are responsible for core and shortcut features, supporting the subnetwork hypothesis.
• SDE Simulation: Numerical simulations of the derived SDEs confirm that increasing data noise increases shortcut bias, while increasing optimization noise decreases it.

Significance and Claims

The paper claims to provide the first formal theoretical characterization of shortcut learning dynamics using evolutionary game theory. Its significance lies in:

1. Unifying Perspectives: It connects feature learning, stochastic optimization, and robustness under a single game-theoretic framework.
2. Explaining Optimization Dynamics: It offers a theoretical explanation for why mini-batch SGD often generalizes better than full-batch GD in the presence of shortcuts, attributing this to the stabilizing effect of optimization noise on the core strategy.
3. Mitigation Insights: The work suggests that mitigating shortcut bias can be approached by manipulating the evolutionary dynamics, such as increasing the effective number of samples following the core strategy (via data augmentation or re-weighting) or amplifying the payoff gap between core and shortcut strategies.
4. Theoretical Boundaries: The authors modestly note that their current framework is limited to supervised learning and coarse-grained modeling, leaving open questions regarding specific model architectures and the interaction coefficient $\gamma$ (which relates to feature disentanglement).

The paper concludes that the balance between shortcuts and core features is not fixed at initialization but is dynamically shaped by the interplay of data noise, optimization stochasticity, and gradient competition.