📊 statistics

Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise

This paper investigates how over-parameterized neural networks simultaneously memorize noisy labels and generalize on arithmetic tasks, revealing that while noise suppresses the output of an internal generalization structure, this structure can be effectively recovered through frequency-based methods even under heavy label noise.

Original authors: Linyu Liu, Pinyan Lu

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Linyu Liu, Pinyan Lu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a very smart, but slightly chaotic, student to do math. You give them a bunch of practice problems, but you accidentally mix in some problems with wrong answers (noise).

Usually, we think a student who memorizes the wrong answers will fail the real test. But this paper discovers something surprising: The student can actually learn the correct math rules and memorize the wrong answers at the same time. They just hide the correct rules inside their brain while their mouth keeps saying the wrong things.

Here is a breakdown of what the researchers found, using simple analogies:

1. The "Double Descent" Surprise

In the old days of machine learning, we thought bigger models were just "bigger sponges" that would soak up every mistake and fail.

The Analogy: Imagine a student who is too small to understand the math. They guess randomly. As they get bigger (more neurons), they start memorizing the specific wrong answers you gave them, so they get worse at the real test.
The Twist: But if you make the student huge (over-parameterized), something magical happens. They become so big that they can hold two things at once: a perfect understanding of the math rules and a list of all the wrong answers.
The Result: The bigger the model, the better they get at the real test, even if the training data is full of lies. They just need the right "study habits" (optimization settings) to unlock this ability.

2. The "Bad News Travels Faster" Phenomenon

One of the most counter-intuitive findings is about when the student learns things.

The Analogy: You might expect a student to learn the correct, logical rules first, and then slowly memorize the weird, random mistakes.
The Reality: The paper found that the student memorizes the wrong answers first. It's like the student hears a loud, confusing shout (the noise) and immediately writes it down because it's easy to grab. The quiet, logical rules (the clean data) take longer to sink in.
Why it matters: This means if you stop training the student too early, they will only know the lies. They need to keep training long enough for the "logic" to catch up and overtake the "memorization."

3. The "Hidden Library" vs. The "Noisy Mouth"

Even when the model is trained on data that is 80% wrong, it still learns the correct math rules inside its brain. The problem is that the "noise" is so loud it drowns out the correct answer when the model speaks.

The Analogy: Imagine a library where the correct books are neatly organized on the shelves (the internal structure), but someone has glued sticky notes with random gibberish all over the covers (the noise). If you just look at the covers, it looks like a mess.
The Discovery: The researchers found a way to "peel off" the sticky notes. By using a frequency filter (a tool that looks for repeating patterns, like a musical tuner), they could isolate the "correct library" from the "gibberish covers."
The Result: Even with 80% noise, they could extract the hidden library and get near-perfect test scores. The rules were there all along; they were just buried under noise.

4. Why You Can't Just "Cut Out" the Bad Parts

The researchers tried a simpler method: they tried to find the specific neurons (brain cells) responsible for the good math and cut out the ones responsible for the bad noise.

The Analogy: Imagine trying to fix a messy room by throwing away the "bad" items and keeping only the "good" ones.
The Failure: This didn't work very well. Why? Because the "good" math and the "bad" noise aren't stored in separate rooms. They are mixed together in the same neurons, like ingredients in a smoothie. You can't just pick out the strawberries without losing the banana.
The Conclusion: To get the good stuff out, you need a sophisticated tool (like the frequency filter mentioned above) that can separate the ingredients based on their "flavor" (mathematical structure), not just by trying to remove specific neurons.

Summary

This paper shows that huge AI models are incredibly robust. Even when fed a diet of lies, they can still learn the truth. They learn the truth slowly and hide it deep inside, while quickly memorizing the lies on the surface. With the right tools, we can dig out that hidden truth, proving that the model didn't just memorize the noise—it actually learned the rules.

Technical Summary: Unveiling Memorization–Generalization Coexistence

Problem Statement

Highly over-parameterized models, such as Large Language Models, demonstrate the capacity to simultaneously memorize noisy labels and generalize well to unseen data. However, the internal mechanisms governing how these two distinct behaviors coexist remain poorly understood. In realistic settings, training data is often contaminated with noise (e.g., mislabeled samples), creating an internal competition between generalization (discovering structural rules applicable to unseen data) and memorization (fitting specific noisy labels that do not reflect underlying rules).

Diagnosing this interaction in complex, real-world tasks is difficult because high test accuracy does not inherently distinguish between learning true rules and exploiting local patterns or shortcuts. To address this, the authors investigate modular arithmetic tasks (specifically addition and multiplication modulo $P$ ) as a controlled experimental environment. These tasks are chosen for their clear distinction between rule-based generalization and point-wise memorization, their exhibit of "grokking" (generalization occurring long after memorization), and the availability of known analytical solutions for specific model configurations.

Methodology

The study employs a two-layer neural network parameterized by $\theta = \{W, U, V, \mu\}$ trained on modular arithmetic tasks with introduced label noise. The noise ratio $\alpha$ is varied, where a subset of the training data is assigned uniform random labels distinct from the true result.

The research addresses three key questions through extensive experiments covering various activation functions (ReLU, Quadratic, GeLU, LeakyReLU), optimizers (Adam, AdamW, Muon, SGD), and regularization strengths:

Q1: Under what conditions does increasing model size improve generalization with noisy labels?
Q2: Can models recover internal structural rules under extreme label noise?
Q3: Can neurons responsible for generalization be separated from those used for memorization?

The methodology involves:

Training Dynamics Analysis: Tracking accuracy on clean vs. noisy subsets during training to observe learning order.
Frequency Filtration (FF): Leveraging the known analytical solution for quadratic activations (which exhibit periodic structures) to decompose ReLU network weights. The authors hypothesize that the generalization signal is carried by the dominant frequency component of the weights, while memorization is encoded in residual frequencies.
Network Partitioning: Attempting to separate the network into generalization and memorization sub-networks using task-agnostic neuron importance scores (Inverse Participation Ratio and Neuron Strength) versus task-specific frequency priors.

Key Findings

1. Scaling and Double Descent

The authors robustly observe the double descent phenomenon in the presence of label noise.

Over-parameterization: Within the over-parameterized regime, increasing model size consistently improves test accuracy, provided optimization and regularization are properly configured.
Regularization: Stronger regularization shifts the model from the over-parameterized regime toward the under-parameterized regime, delaying the second descent.
Activation Functions: ReLU models show non-decreasing test accuracy in the over-parameterized regime. In contrast, models with Quadratic or GeLU activations exhibit ceilings in test accuracy as model size increases, potentially due to symmetry constraints on how noise is absorbed.

2. Counter-Intuitive Learning Dynamics

A significant finding is that noisy labels are memorized faster than clean labels during the initial training epochs, regardless of the optimizer (AdamW, Adam, Muon) or noise ratio.

This inversion of the typical learning order is attributed to full-batch training dynamics. The noisy subset, being smaller, receives a stronger per-sample optimization signal from the full-batch gradient compared to the larger clean subset, causing its accuracy to rise earlier.
This phenomenon was also observed in MNIST experiments, suggesting it is a broader property of early full-batch optimization under explicit label corruption.

3. Coexistence of Rules and Noise

Even under extreme label noise (up to 80%), models internally learn the correct algebraic rule, though this rule is suppressed in the raw output by the need to interpolate noisy labels.

Frequency Filtration: By isolating the dominant frequency component of the weights (aligned with the analytical solution for quadratic activations), the authors can extract a sub-network that achieves near-perfect test accuracy, even when the full model's output is heavily corrupted.
Residuals: The residual frequency components primarily retain the ability to fit noisy labels but contribute little to generalization.
Activation Independence: Replacing the ReLU activation with Quadratic or Reverse ReLU in a trained network preserves or improves generalization, indicating the learned weights already encode a symmetric rule representation compatible with these functions.

4. Limits of Task-Agnostic Separation

The paper investigates whether generalization and memorization can be separated at the neuron level without relying on task-specific priors (like Fourier structure).

Neuron Selection: Using metrics like Inverse Participation Ratio (IPR) or Neuron Strength (Str.), the authors prune the network to select neurons for generalization. While this improves generalization slightly, the gain is significantly weaker than that achieved by Frequency Filtration.
Entanglement: The results suggest that the generalizable structure is not localized to a small subset of neurons but is diffusely encoded across the network and deeply entangled with components used for noise memorization. Task-agnostic pruning cannot fully extract the rule structure learned by the model.

Significance and Claims

The paper claims to provide a rigorous, verifiable boundary between structural generalization and unstructured memorization in over-parameterized learning. Its primary contributions are:

Mechanistic Insight: It demonstrates that over-parameterized models can internally learn underlying rules even under explicit label noise, but these rules are masked by the necessity to fit noise.
Frequency-Based Decoupling: It introduces a method (Frequency Filtration) that successfully disentangles generalization from memorization in modular arithmetic tasks by exploiting the task's structural priors, recovering near-perfect accuracy from heavily corrupted training.
Limitation of Post-Hoc Pruning: It highlights a fundamental limitation of current neuron-level tools. While task-agnostic neuron selection offers marginal improvements, it fails to fully extract the generalizable structure because the rule is distributed and entangled across the network.
Broader Implications: The authors suggest that future learning paradigms may need to explicitly isolate memorization from logical inference during training, and that effective post-hoc knowledge extraction may require domain-specific structural priors rather than generic importance scores.

The authors acknowledge that their setup abstracts away the complex variability of natural vision or language tasks. However, they argue that this controlled setting is necessary to rigorously study the coexistence of memorization and generalization, providing foundational evidence that these processes are deeply intertwined in over-parameterized networks.