🔢 mathematics

Uncertain but Useful: Leveraging CNN Training Variability into Data Augmentation

This paper demonstrates that the numerical uncertainty inherent in training the FastSurfer deep learning model for neuroimaging not only impacts reliability but can also be effectively harnessed as a data augmentation technique through ensembling to improve downstream brain age regression performance.

Original authors: Inés Gonzalez-Pepe, Vinuyan Sivakolunthu, Yohan Chatelain, Tristan Glatard

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Inés Gonzalez-Pepe, Vinuyan Sivakolunthu, Yohan Chatelain, Tristan Glatard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: When Computers "Guess" Differently

Imagine you are baking a cake using a very precise recipe (a Deep Learning model). You expect that if you use the exact same ingredients and the exact same oven settings, you will get the exact same cake every time.

However, this paper discovered that with modern AI brain scanners, that's not quite true. Even when you use the exact same brain scan and the exact same recipe, the computer might bake slightly different "cakes" (segmentations) depending on tiny, invisible fluctuations in how the math is calculated.

The researchers asked two big questions:

Is this "baking inconsistency" a problem? (Does it make our results unreliable?)
Can we use this inconsistency to our advantage? (Can we turn a bug into a feature?)

Part 1: The Problem – "The Shaky Hand"

The researchers compared two ways of mapping the brain:

FreeSurfer: The old-school, traditional method (like a master carpenter using a hand saw).
FastSurfer: The new, fast Deep Learning method (like a high-tech laser cutter).

The Discovery:
They found that when the new AI (FastSurfer) is learning (training), it is actually more shaky than the old method, especially in the outer layers of the brain (the cortex).

The Analogy:
Imagine two artists trying to trace a map of a city.

The Old Artist (FreeSurfer) draws the map once. If they make a tiny mistake, it's consistent.
The New Artist (FastSurfer) is learning to draw. Every time they practice, their hand shakes just a tiny bit differently because of how their brain processes the numbers. Sometimes they draw a street a millimeter to the left; other times, a millimeter to the right.

The paper found that this "hand shake" in the new AI is so strong that it can change the final map more than the old method does. This matters because if a doctor relies on that map to measure brain size, a tiny shift could change a diagnosis from "healthy" to "at risk."

Part 2: The Shortcut – "The Magic Dice"

To measure this "hand shake," the researchers used a complex, slow mathematical tool called Monte Carlo Arithmetic. It's like checking every single grain of sand in a hourglass to see if any are slightly the wrong size. It's accurate but takes forever.

The Discovery:
They found a much faster way to measure this instability. They simply changed the Random Seed (think of this as rolling a different die) at the start of the training.

The Analogy:
Instead of checking every grain of sand, they just asked the artist to start the drawing with a different "mood" (a different random seed). They found that the resulting "shaky" maps looked almost exactly the same as the ones made by the super-slow, complex math tool.

Why this is great: It's like realizing you don't need a microscope to see the wobble in your hand; you just need to shake it a few different ways and look at the result. It's a cheap, fast way to see how unstable the AI is.

Part 3: The Solution – "The Choir Effect"

Usually, when a computer makes different results, we think that's a bug. But the researchers asked: What if we use all those different versions to make a better answer?

The Discovery:
They trained 15 different versions of the AI, each with a different "mood" (random seed). Then, instead of picking just one, they combined all 15 maps together (an "ensemble").

The Analogy:
Imagine you are trying to guess the temperature outside.

Method A: You ask one person. They might be slightly off.
Method B: You ask 15 people who all have slightly different watches. You take the average of their answers.
The Result: The average of the 15 guesses is usually much closer to the real temperature than any single guess.

In this paper, they used this "Choir Effect" to predict Brain Age (estimating how old a brain is based on its shape).

They found that using the combined "shaky" maps made the age prediction much more accurate.
They compared this to other methods that try to fake new data (like a photocopier making blurry copies). The "Choir" method worked better because the 15 maps were all based on real brains, just viewed slightly differently. The fake data looked "unrealistic" and didn't help as much.

Summary of Key Takeaways

AI is Unstable: The new, fast AI brain scanners are actually more sensitive to tiny math errors during training than the old, slow methods. This could affect medical results if not accounted for.
Random Seeds are a Good Proxy: You don't need complex math to measure this instability; just changing the random starting number (the seed) gives you a clear picture of how much the AI wobbles.
Embrace the Wobble: Instead of fighting the instability, the researchers used it. By combining multiple slightly different versions of the AI, they created a "super-model" that predicts brain age better than standard methods, without needing to collect any new patient data.

In short: The paper shows that the "noise" in AI training is real and dangerous, but if you listen to the whole choir instead of just one singer, that noise can actually help you hear the truth more clearly.

Technical Summary: Uncertain but Useful: Leveraging CNN Training Variability into Data Augmentation

Problem Statement

While deep learning (DL) has revolutionized neuroimaging by offering state-of-the-art performance with reduced computation times, the numerical uncertainty inherent to the DL training phase remains largely unexplored. Unlike inference, training involves iterative, high-dimensional optimization where small numerical perturbations can accumulate across iterations, potentially steering models toward different solutions even on identical data. Although numerical uncertainty is well-documented in traditional neuroimaging pipelines (e.g., FreeSurfer), its behavior and impact during DL training are not fully understood. This gap is critical as neural networks increasingly replace classical toolkits, and recent work suggests that comparable numerical noise in traditional pipelines can distort clinical findings.

Methodology

The authors employed a multi-faceted approach to quantify and leverage training-time variability in the FastSurfer segmentation model:

Numerical Perturbation via Monte Carlo Arithmetic (MCA):
- To simulate hardware and software variability, the authors applied MCA, a stochastic arithmetic method that introduces random perturbations in floating-point operations at runtime.
- Using Fuzzy PyTorch, they integrated stochastic arithmetic directly into the PyTorch execution pipeline, applying a perturbation of 1 unit of least precision (ulp) to simulate virtual precision errors.
- For comparison, MCA was also applied to the traditional FreeSurfer v7.3.2 pipeline using "fuzzy libmath" on CPU.
Random Seed Perturbation as a Proxy:
- To establish a practical baseline, models were trained with 15 different random seeds on GPU. These seeds controlled pseudorandom processes (weight initialization, dropout) while keeping data shuffling fixed to isolate model (epistemic) uncertainty.
- The structural similarity between seed-induced variability and MCA-induced variability was assessed to validate seeds as a low-cost proxy for numerical uncertainty.
Ensembling as Data Augmentation:
- The authors reframed training-time instability as a data augmentation technique. Multiple FastSurfer models were trained with different seeds, and their outputs were ensembled to generate 15 distinct yet anatomically valid segmentations per subject.
- These ensemble-derived Region of Interest (ROI) volumes were used as features for downstream brain age regression tasks using Random Forest (RF), Support Vector Machine (SVM), and Gradient Boosting (GB) models.
- This approach was compared against synthetic data augmentation methods, specifically a Gaussian Copula synthesizer and variational autoencoders (the latter excluded due to unrealistic distributions).
Evaluation Metrics:
- Variability: Measured using the minimum Sørensen-Dice score across pairs of repetitions to capture worst-case consistency.
- Downstream Performance: Evaluated via Mean Absolute Error (MAE) in brain age regression on internal datasets (FastSurfer training set) and an external dataset (Healthy Brain Network, HBN).

Key Results

Training-Time Variability Exceeds Traditional Methods in Cortical Regions:
- Quantification revealed that FastSurfer's training-time variability meets or exceeds that of FreeSurfer, particularly in cortical structures. While subcortical variability was largely comparable, all cortical regions exhibited significantly greater variability with FastSurfer.
- This suggests that FastSurfer is subject to the same reproducibility concerns previously identified in FreeSurfer, with potential implications for downstream clinical results.
Seed Perturbation as a Valid Proxy:
- Regional variability rankings between MCA and random seed perturbations were strongly correlated (Spearman $\rho = 0.877$ ).
- Despite MCA being conducted on CPU and seeds on GPU, the strong structural agreement indicates that random seeds effectively capture the regional patterns of instability, providing a computationally feasible alternative to exhaustive MCA.
Numerical Ensembling Improves Downstream Performance:
- Models trained with different seeds achieved performance comparable to a fixed IEEE baseline, confirming that the observed variability does not degrade the core model capability.
- Data Augmentation Efficacy: Numerical ensembling consistently improved brain age regression performance. Increasing the number of ensemble repetitions led to significant monotonic improvements in MAE for RF, SVM, and GB models on both internal and external datasets.
- Comparison to Synthetic Methods: Numerical ensembling outperformed synthetic augmentation (Gaussian Copula). While synthetic methods showed weak or non-significant correlations with performance improvements, numerical ensembling provided robust gains. The authors attribute this to the fact that ensemble members represent authentic neuroanatomical variations of real MRI scans, whereas synthetic methods struggle to learn realistic distributions from limited data.

Significance and Claims

The paper positions numerical uncertainty during DL training as a substantive factor in neuroimaging reliability with measurable consequences for downstream tasks. The authors make two primary claims:

Risk Assessment: The numerical uncertainty inherent in FastSurfer training is at least as significant as that in the traditional FreeSurfer pipeline, particularly in cortical regions. This necessitates systematic investigation into uncertainty propagation from DL-based pipelines to downstream clinical analyses.
Opportunity for Augmentation: Training-time variability, characterized effectively by random seed perturbations, can be constructively harnessed as a data augmentation technique. By leveraging ensembling, researchers can generate diverse, anatomically realistic representations without additional data collection, thereby improving the robustness of downstream regression tasks compared to synthetic data generation.

The work concludes that numerical uncertainty should not merely be viewed as a source of error to be minimized, but as a resource that, when properly characterized and leveraged, can enhance the reliability and performance of neuroimaging applications.