🤖 machine learning

Influence Dynamics and Stagewise Data Attribution

This paper introduces a stagewise data attribution framework grounded in singular learning theory, revealing that training data influence is dynamic and non-monotonic, with specific shifts and sign flips that align with the progressive learning of semantic hierarchies in both toy models and large language models.

Original authors: Jin Hwa Lee, Matthew Smith, Maxwell Adam, Jesse Hoogland

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jin Hwa Lee, Matthew Smith, Maxwell Adam, Jesse Hoogland

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Timing is Everything

Imagine you are teaching a child to sort a giant box of mixed-up toys.

The Old Way (Static View): Traditional methods of analyzing training data assume that a specific toy (like a red block) always has the same effect on the child's learning, no matter when you show it to them. It's like saying, "The red block is always helpful for learning colors."
The New Way (Stagewise View): This paper argues that this is wrong. In reality, the effect of that red block changes dramatically depending on the stage of the lesson.
- Early Stage: Showing the red block helps the child learn "things are red."
- Middle Stage: If you keep showing the red block while trying to teach the difference between "animals" and "vehicles," it might actually confuse the child.
- Late Stage: Later, when teaching fine details, that same red block might become crucial again.

The authors call this Stagewise Data Attribution. They argue that to understand how AI learns, we can't just look at the final result; we have to watch the movie of the learning process, not just the final frame.

The Problem: Why the Old Tools Fail

The paper explains that current tools for measuring "influence" (how much one piece of data changes the model) are built for simple, predictable systems. They assume the learning process is a smooth, straight line.

But neural networks (the brains behind AI) are messy. They are like a landscape with many valleys and hills that shift around. The authors use a theory called Singular Learning Theory to explain that AI learning isn't smooth; it happens in stages or phases.

Analogy: Think of learning to ride a bike. At first, you wobble (Phase 1). Then, you suddenly find your balance (a "phase transition"). After that, you learn to turn (Phase 2). The way a pebble on the road affects you is totally different when you are wobbling compared to when you are turning. The old tools treat the pebble's effect as if it never changes, which is a disaster for understanding complex AI.

The Solution: A Dynamic Lens

The authors propose a new tool called the Bayesian Influence Function (BIF).

The Metaphor: Instead of taking a single snapshot of the AI's brain, this tool takes a video. It tracks how the "importance" of a specific data point rises and falls, flips from helpful to harmful, and spikes at critical moments.

They predict three specific things will happen during these learning stages:

Sign Flips: A piece of data can be a "friend" (helpful) at one moment and a "foe" (harmful) the next.
Sharp Peaks: Influence can suddenly spike to a massive level right when the model is switching gears (a phase transition).
Non-Monotonicity: The influence doesn't just go up or down; it wiggles, peaks, and drops in complex patterns.

Proof 1: The Toy Model (The "Animal" Lesson)

To test this, the authors used a simple, controlled AI model trained on a dataset of animals and plants.

The Setup: The model had to learn a hierarchy: First, "Living vs. Non-living." Second, "Animal vs. Plant." Third, "Mammal vs. Bird." Finally, "Dog vs. Cat."
The Discovery: They watched how the model viewed a "Dog" sample over time.
- Early on: The "Dog" sample helped the model learn about "Sparrows" (both are animals). The influence was positive (helpful).
- Later: As the model tried to distinguish "Mammals" from "Birds," the "Dog" sample actually started to hurt the learning of "Sparrows." The influence flipped signs.
- The Peak: The influence spiked exactly when the model was struggling to separate "Mammals" from "Birds."
The Takeaway: The data point didn't change; the stage of learning changed, and that altered the data's role completely.

Proof 2: Real Language Models (The "Grammar" Lesson)

The authors then tested this on large language models (like the ones that write text) to see if it holds up in the real world. They looked at how the model learned specific structural patterns, like induction (recognizing that if "A" follows "B" once, it might follow "B" again).

The Discovery: They found that the influence of "induction tokens" (words that help the model spot patterns) didn't just slowly increase.
- It stayed low.
- Then, at a specific moment in training (around 30,000 steps), it spiked dramatically.
- This spike perfectly matched the moment the model was known to "discover" the induction circuit (the internal mechanism for pattern matching).
The Takeaway: The model didn't learn grammar in a straight line. It had a specific "aha!" moment where certain data became incredibly influential, and that influence faded or changed afterward.

Why This Matters

The paper concludes that we need to stop asking, "Which data points are the most important?" and start asking, "When are these data points the most important?"

The Analogy: Imagine a conductor leading an orchestra. If you ask, "Which instrument is the most important?", the answer changes every second. The violin is crucial for the melody, but the drums are crucial for the rhythm. If you only looked at the final recording, you might miss the fact that the drums were silent for the first half of the song.
The Goal: By understanding these "stages," we can better understand how AI learns, debug why it makes mistakes at specific times, and potentially design better training schedules (like a curriculum) that feed the right data at the right moment.

In short: AI learning is a story with chapters, not a single sentence. To understand the story, you have to know which chapter you are in.

Technical Summary: Influence Dynamics and Stagewise Data Attribution

Problem Statement

Current Training Data Attribution (TDA) methods, particularly Influence Functions (IFs), operate under the assumption that the influence of a training sample on a model's behavior is static and global throughout the training process. This perspective, inherited from the analysis of regular statistical models, fails to account for the developmental dynamics of deep neural networks. Neural networks possess degenerate loss landscapes with non-isolated critical points and non-invertible Hessians, violating the regularity assumptions required for classical IFs. Consequently, existing methods cannot capture how the role of a data sample evolves as the model learns, ignoring the fact that a sample helpful in early training may become harmful or irrelevant in later stages.

Methodology

The authors propose a framework for Stagewise Data Attribution grounded in Singular Learning Theory (SLT). The methodology involves three primary components:

Theoretical Framework (SLT & Bayesian Influence):
- The paper models the learning trajectory of stochastic optimizers (like SGD) using an idealized Bayesian learning process.
- It replaces classical Influence Functions with Bayesian Influence Functions (BIF). Unlike classical IFs which rely on the inverse Hessian at a single minimum, BIFs measure the change in the posterior expectation of an observable. This is defined as the negative covariance between the sample's loss and the observable: $BIF(z_i, \phi) = -Cov_{p(w|D)}(\ell_i(w), \phi(w))$ .
- This formulation is "Hessian-free," well-defined on degenerate landscapes, and applicable at any point in the training trajectory, not just at convergence.
Analytical Decomposition:
- Using the Law of Total Covariance, the authors decompose influence into "Within-Phase" and "Between-Phase" components.
- They predict that during phase transitions (where the posterior mass splits between two distinct solutions, $U$ and $V$ ), the "Between-Phase" term dominates. This leads to non-monotonic dynamics, including sharp peaks in influence and sign flips, particularly when the posterior probabilities of the phases are roughly equal ( $\pi_U \approx \pi_V \approx 0.5$ ).
Empirical Validation:
- Toy Model: A 2-layer deep linear network trained on a hierarchical semantic dataset (Saxe et al., 2019a). The authors use Stochastic Gradient Langevin Dynamics (SGLD) to estimate posteriors and compute local BIFs. They validate these findings analytically and via Leave-One-Out (LOO) retraining experiments.
- Large-Scale Language Models: Experiments conducted on Pythia language models trained on The Pile dataset. The authors compute per-token BIFs to track influence dynamics of specific syntactic classes (e.g., delimiters, induction patterns) across training checkpoints.

Key Contributions

Developmental Framework for Influence: The paper introduces a theoretical framework connecting influence functions to stagewise development. It posits that influence is not static but changes non-monotonically, exhibiting sign flips and sharp peaks at developmental phase transitions.
Validation in Controlled Systems: The authors provide a concrete proof-of-concept in a hierarchical feature-learning model, demonstrating that dynamic shifts in influence directly map to the sequential learning of the data's hierarchical structure (e.g., distinguishing animals vs. plants, then mammals vs. birds).
Scalable Application in Language Models: The paper demonstrates that these phenomena occur at scale. It shows that token-level influence functions for structural elements (like delimiters and induction patterns) undergo non-monotonic changes that align with known developmental stages, such as the formation of induction circuits.

Results

Non-Monotonic Dynamics: In the toy model, the influence of a "dog" sample on a "sparrow" sample was found to be negative (helpful) during the early phase of learning broad categories (animal vs. plant) but became positive (harmful) during later phases of learning finer distinctions (mammal vs. bird).
Correlation with Phase Transitions: Peaks in influence were observed to coincide exactly with the branching points in the model's hidden representation (visualized via Multi-Dimensional Scaling), marking the transition between learning different levels of the semantic hierarchy.
Causal Windows: Ablation experiments revealed that removing data precisely during the time window where BIF assigns peak influence causes the largest loss difference, confirming that influence is time-dependent and that specific data points drive learning only during specific developmental stages.
Language Model Findings: In Pythia models, the influence of induction-pattern tokens showed a sharp rise and subsequent fall, aligning with the known timeline of induction circuit formation (peaking around 30k steps). Similarly, the relationship between left and right delimiters exhibited sign flips, indicating a qualitative shift in how the model processes text structure.

Significance and Claims

The paper argues for a fundamental shift from point-wise to stagewise data attribution.

Beyond Static Scores: The authors claim that assigning a single, global influence score to a data point at the end of training provides an incomplete and potentially misleading picture. A sample's importance is inseparable from the developmental stage of the model.
Mechanism for Implicit Curricula: The findings suggest a mechanism for how neural networks adopt an "implicit curriculum." Different data points become influential at different moments, effectively creating a self-organizing learning schedule where the model prioritizes specific structures at specific times.
Limitations and Future Directions: The authors acknowledge that their framework relies on the link between Bayesian learning (SLT) and non-equilibrium SGD dynamics, which remains a theoretical gap. They also note that future work should move from behavioral attribution to a mechanistic account, understanding how these dynamic influences shape internal circuits and features.

The paper concludes that understanding when and why data matters is as critical as understanding which data matters, offering a new lens for debugging, interpreting, and potentially steering the learning process of deep neural networks.