📊 statistics

On the Epistemic Uncertainty of Overparametrized Neural Networks

This paper challenges the conventional view that epistemic uncertainty vanishes with more data by demonstrating that in overparametrized neural networks, non-identifiability due to symmetries and redundant representations leads to persistent parameter uncertainty even when the underlying function is fully identified, a phenomenon the authors analyze theoretically and validate empirically in one-hidden-layer ReLU networks.

Original authors: David Rügamer

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: David Rügamer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Many Keys, One Door" Problem

Imagine you have a very complex lock (the real-world problem) and a massive keyring with thousands of keys (the neural network). Usually, we think that if we have enough data, we can find the one perfect key that opens the lock, and we will be 100% sure we found it.

This paper argues that for modern, "overparametrized" neural networks (networks with way more keys than necessary), this isn't true. Even with infinite data, you might never know exactly which specific key you are holding. You only know that the door is open.

The authors call this Epistemic Uncertainty (uncertainty about what we know). They show that even when the network knows the answer perfectly, it remains confused about how it arrived at that answer because there are many different ways to build the same solution.

Analogy 1: The Orchestra Conductor (Permutation Symmetry)

Imagine a conductor leading an orchestra. The music (the prediction) is perfect.

The Problem: The violin section has 100 players. If Player #1 swaps seats with Player #50, the music sounds exactly the same.
The Confusion: If you ask the conductor, "Who is playing the first violin?" and they say, "It's Player #1," you might think they are 100% sure. But actually, Player #50 could have been playing that role, and the music would still be perfect.
The Paper's Finding: In standard statistics, we assume that with enough practice (data), the conductor will know exactly who is sitting where. But in these giant networks, the conductor can never be sure of the seating chart, even if the music is flawless. The uncertainty about who is playing (the parameters) remains, even though the song (the function) is known perfectly.

Analogy 2: The Pizza Slicing (Continuous Non-Identifiability)

Now, imagine the network is even bigger than necessary. It has extra "neurons" (extra chefs) that aren't strictly needed.

The Scenario: You need to bake a pizza that requires exactly 100 grams of cheese.
The Confusion:
- Scenario A: One chef puts all 100g on the pizza.
- Scenario B: Two chefs put 50g each.
- Scenario C: Ten chefs put 10g each.
- Scenario D: Chef A puts 99g, Chef B puts 1g.
The Paper's Finding: The network can achieve the exact same pizza (the same prediction) by splitting the "cheese" (the weight) among the extra chefs in infinite different ways.
- Unlike the orchestra where people just swap seats, here the chefs are constantly arguing over how to split the cheese.
- The paper proves that even with infinite data, the network doesn't settle on one specific way to split the cheese. Instead, it wanders around a "manifold" (a smooth surface of possibilities) where the total amount of cheese is always 100g, but the individual amounts keep changing.

Why Does This Matter? (The "Why Common Measures Fail" Section)

Usually, when scientists measure how "uncertain" a model is, they look at how much the output changes.

The Old View: "If the model gives the same answer every time, it's not uncertain."
The Paper's View: "That's wrong. The model might give the same answer, but the internal gears (the weights) are spinning wildly in different directions."

The authors show that if you only look at the final answer (the pizza), you miss the fact that the internal machinery is chaotic. This matters if you need to know how the model works (for things like debugging, compression, or understanding which features are important), not just what the answer is.

What Did They Do?

The Theory: They used math to prove that for simple neural networks (ReLU networks), this "splitting" and "swapping" creates a permanent fog of uncertainty inside the model's brain, even when the model is perfect at its job.
The Math: They described this fog using shapes called manifolds. Think of it like a smooth, flat sheet of paper floating in a 3D room. The model's weights can slide anywhere on that sheet without changing the result.
The Experiments: They built these networks and ran them on computers. They watched the "weights" (the internal numbers) move.
- Result 1: The networks got better at predicting as they saw more data.
- Result 2: But the internal weights never stopped moving around. They kept sliding along those "sheets" (manifolds) and swapping places, proving that the uncertainty about the internal structure never went away.
The Sampling Issue: They also looked at how computers try to "explore" these networks (using a method called MCMC). They found that if you start a computer simulation in one specific "seating arrangement" (one permutation), it usually gets stuck there and never jumps to the other equivalent arrangements, even though they are all valid. This means standard computer methods might miss the full picture of the uncertainty.

The Takeaway

The paper claims that overparametrized neural networks have a hidden, permanent uncertainty about their own internal structure.

Function Space (The Output): The model becomes certain. It knows the answer.
Parameter Space (The Internal Weights): The model remains uncertain. It doesn't know which specific combination of numbers it is using to get that answer, because there are millions of equally good combinations.

This is like a master chef who can cook a perfect dish, but if you ask them, "Did you use 2 teaspoons of salt or 1 teaspoon of salt and 1 teaspoon of soy sauce?" they can't tell you, because both recipes taste exactly the same. The paper argues we need to stop pretending the chef knows the exact recipe just because the food tastes good.

Technical Summary: On the Epistemic Uncertainty of Overparametrized Neural Networks

Problem Statement

Epistemic uncertainty in machine learning is traditionally defined as reducible uncertainty that vanishes as data size increases, often equated with predictive variability. This perspective relies on the assumption of parameter identifiability, where the posterior distribution concentrates around a unique parameter vector (or a point mass) as $n \to \infty$ . However, in overparametrized neural networks, parameters are typically non-identifiable due to symmetries (e.g., permutation invariance) and redundant representations.

The core problem addressed is that while the underlying function $f^*$ may be fully identified, the parameter space contains substantial uncertainty that persists even with infinite data. Classical uncertainty decompositions, which focus on predictive variance, fail to capture this "residual" uncertainty because they are invariant to parameter variations that leave the function unchanged. Consequently, there is a mismatch between the structural properties of overparametrized models (where the posterior does not collapse to a point in parameter space) and standard uncertainty quantification methods.

Methodology

The authors analyze epistemic uncertainty through the lens of non-identifiability using a one-hidden-layer ReLU network architecture:
$f_w(x) = w_2^\top \phi(W_1 x)$
where $\phi(\cdot) = \max(0, \cdot)$ . The study assumes a regularized loss objective equivalent to a Bayesian posterior with a Gaussian prior.

The methodology proceeds in three stages:

Theoretical Characterization: The authors distinguish between two sources of non-identifiability:
- Permutation Symmetry: In correctly specified models ( $M = M^*$ ), non-identifiability arises from permuting hidden neurons. The posterior converges to a mixture of Dirac measures over these permutations.
- Continuous Splitting (Manifold Assignment): In overparametrized models ( $M > M^*$ ), true neurons can be represented by continuous combinations of multiple redundant neurons. This induces a "splitting manifold" $\mathcal{M}_\varsigma$ where weights can be redistributed continuously while maintaining the same function and regularization cost.
Posterior Derivation: The paper derives the exact form of the induced posterior on these manifolds. It establishes that for a fixed assignment map $\varsigma$ (mapping model neurons to true neurons), the splitting coefficients follow a symmetric Dirichlet distribution. The authors define the posterior on the manifold as the weak limit of the ambient posterior conditioned on an $\epsilon$ -tube around the manifold as $\epsilon \to 0$ .
Empirical Validation: The theoretical findings are validated using Bayesian Neural Networks (BNNs) with Gaussian priors and likelihoods. Inference is performed using Stochastic Gradient Langevin Dynamics (SGLD) and the No-U-Turn Sampler (NUTS). The study analyzes posterior samples to verify convergence properties, switching rates between permutation chambers, and the distribution of splitting coefficients.

Key Contributions

Reframing Epistemic Uncertainty: The paper argues that epistemic uncertainty in overparametrized models includes a persistent component due to non-identifiability that does not vanish with increasing data, even when the predictive function is perfectly identified.
Characterization of Residual Uncertainty:
- Discrete Source: In correctly specified models, uncertainty manifests as a mixture of Dirac measures over permutation orbits.
- Continuous Source: In overparametrized models, uncertainty manifests as a non-degenerate distribution over splitting manifolds. The authors derive that splitting coefficients follow a Dirichlet distribution $\text{Dir}(\alpha, \dots, \alpha)$ with $\alpha = (p+1)/2$ .
Posterior Moments and Geometry: The authors derive explicit formulas for the posterior moments of weights under overparametrization. They show that while the total contribution of a group of redundant neurons remains fixed, the individual contribution of each neuron shrinks ( $O(M^{-1/2})$ in mean, $O(M^{-1})$ in covariance) as the network width $M$ increases.
Sampling Dynamics: The paper demonstrates that standard MCMC samplers (under specific continuity assumptions) are almost surely confined to a single permutation chamber or a single assignment manifold interior. They do not naturally traverse the boundaries between different symmetry-related regions, meaning a single chain explores only a subset of the full posterior support.

Results

Convergence vs. Variance: Empirical results confirm that while predictive performance (RMSE, LPPD) converges to the true function as $n$ increases, the trace of the weight covariance matrix remains non-zero due to permutation and splitting symmetries.
Manifold Structure: In overparametrized settings ( $M > M^*$ ), posterior samples align with the theoretical prediction that splitting coefficients follow a Beta distribution (marginal of Dirichlet). The empirical distributions of weight contributions closely match the theoretical Dirichlet marginals.
Sampling Behavior: Experiments with NUTS and SGLD show that chains remain confined to single permutation chambers. The "switch rate" between chambers is negligible, and the "mean minimum margin" indicates chains reside deep within specific chambers, rarely crossing the measure-zero boundaries where neurons become exchangeable.
Practical Impact: The paper illustrates that ignoring non-identifiability can lead to distorted estimates in downstream tasks. In continual learning (EWC), non-identifiability-induced variance can falsely inflate parameter importance, leading to overly restrictive updates. Correcting for this variance improves adaptation. Similarly, standard convergence diagnostics (like $\hat{R}$ ) in parameter space may remain inflated due to non-identifiable directions, even when function-space convergence is achieved.

Significance and Claims

The paper claims to provide a necessary correction to the standard view of epistemic uncertainty in deep learning. It asserts that:

Parameter Uncertainty Persists: Unlike the classical view where uncertainty vanishes with data, parameter uncertainty in overparametrized networks is irreducible due to structural symmetries.
Function vs. Parameter Space: While working in function space mitigates these issues, procedures operating directly on weights (e.g., interpretability, model compression, continual learning, or sampling-based inference) must account for this residual uncertainty to avoid misleading conclusions.
Limitations of Current Heuristics: Practical heuristics like deep ensembles may "average out" some effects of non-identifiability, but they do not eliminate the underlying geometric structure of the posterior. The paper suggests that understanding the geometry of the posterior (specifically the manifolds induced by overparametrization) is crucial for accurate uncertainty quantification in parameter space.

The authors remain modest regarding the scope, noting that their formal results are derived for one-hidden-layer ReLU networks with Gaussian priors. They acknowledge that deeper networks introduce additional layer-wise non-identifiabilities not covered by their theory, and that the specific Dirichlet splitting law depends on the choice of Gaussian prior/L2 regularization. However, they argue that the mechanisms of permutation and splitting symmetries are fundamental to overparametrized models and likely extend to broader architectures.