🤖 machine learning

Fuzzy PyTorch: Rapid Numerical Variability Evaluation for Deep Learning Models

This paper introduces Fuzzy PyTorch, a scalable and efficient framework that integrates stochastic and up-down rounding modes via Verificarlo to rapidly evaluate numerical variability in deep learning models with significantly reduced runtime compared to existing tools, all while maintaining model performance.

Original authors: Inés Gonzalez-Pepe, Hiba Akhaddar, Tristan Glatard, Yohan Chatelain

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Inés Gonzalez-Pepe, Hiba Akhaddar, Tristan Glatard, Yohan Chatelain

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are baking a massive, complex cake (a Deep Learning model) using a very precise digital scale. In the real world, if you measure flour, sugar, and eggs with perfect precision, you get the same cake every time. But in the world of computers, the "scale" (the floating-point arithmetic) has tiny limitations. It can't hold infinite decimals, so it has to round numbers off. Usually, it rounds in a standard way, but sometimes, because of how the computer's brain works, that tiny rounding decision can change the final result slightly.

For a long time, scientists worried: "If we change the rounding just a tiny bit, does the cake taste different? Does the AI make a different decision?" To find out, they had to bake the same cake thousands of times, changing the rounding rules slightly each time to see if the results stayed stable.

The Problem: The Slow Cook
The paper explains that the existing tools for doing this "baking test" were incredibly slow. Imagine trying to taste-test your cake by baking it 100 times, but every time you bake it, the oven takes 100 times longer than usual because it's checking every single ingredient with a magnifying glass. If you have a small cake, it takes a day. If you have a giant wedding cake (a massive AI model with millions of parameters), it would take years. This made it impossible to test modern, huge AI models.

The Solution: Fuzzy PyTorch
The authors created a new tool called Fuzzy PyTorch. Think of this as a "Super-Fast Rounding Switch" that you can flip on your kitchen scale.

Instead of the old, slow method of checking every single number, Fuzzy PyTorch uses two clever tricks (called rounding modes) to speed things up:

Stochastic Rounding (The "Gentle Nudge"): This is like looking at a number that is halfway between two marks on the scale. Instead of always rounding up or down, it flips a coin. Heads, you round up; tails, you round down. This keeps the average result accurate but introduces the tiny "wobble" needed to test stability.
Up-Down Rounding (The "Fast Flip"): This is a new, even faster trick invented by the authors. It's like taking a number that has already been rounded and flipping a coin to see if you nudge it up or down by the smallest possible amount. It's less precise than the first method, but it is incredibly fast.

How It Works
The team built a library called PRISM (Probabilistic Rounding with Instruction Set Management). Imagine PRISM as a high-speed assembly line worker who can grab a number, flip a coin, and round it in a fraction of a second, using the computer's built-in speed boosts (vectorized instructions).

They hooked this up to PyTorch, a popular framework for building AI. Before this, testing these models was like trying to run a marathon while wearing a backpack full of bricks. With Fuzzy PyTorch, they took the bricks off.

The Results: Speed and Safety
The paper tested this new tool on three different "cakes":

MNIST: A simple task of recognizing handwritten digits (a small, easy cake).
FastSurfer: A complex task of mapping the human brain from MRI scans (a medium-sized, detailed cake).
WavLM: A task of identifying Parkinson's disease from voice recordings (a very large, complex cake).

The Findings:

Speed: Fuzzy PyTorch was 5 to 60 times faster than the previous best tool (Verrou). In some cases, a test that would have taken a week now takes a few hours.
Accuracy: Even though it was faster, the results were just as reliable. The "wobble" it introduced was similar to the old tools, meaning researchers could trust the data.
Scalability: Because it is so fast, it can now handle the massive AI models of today, which the old tools simply couldn't touch.

Why It Matters
The paper concludes that Fuzzy PyTorch is like giving researchers a turbo-charged oven. It allows them to quickly check if their AI models are robust and reliable without waiting months for results. It doesn't just save time; it makes it possible to study the "numerical variability" (the tiny rounding errors) of the biggest, most complex AI systems in the world, ensuring they work correctly before they are used in real life.

The authors also note that they found some "instability" in the background of brain scans during testing, which they used to improve the software later, proving that finding these tiny errors can actually lead to better, faster AI.

Technical Summary: Fuzzy PyTorch: Rapid Numerical Variability Evaluation for Deep Learning Models

Problem Statement

Deep learning (DL) models are increasingly deployed in high-stakes domains such as medical imaging, remote sensing, and scientific simulations. However, these models rely on numerical computations sensitive to floating-point precision and rounding modes. While understanding numerical variability is essential for ensuring robustness and reproducibility, existing tools for assessing this variability face significant scalability challenges.

Traditional exact methods (e.g., interval arithmetic, formal verification) often require extensive code modifications and do not scale to complex DL workloads. Conversely, stochastic arithmetic tools, which introduce controlled random perturbations to estimate variability, are often prohibitively slow. State-of-the-art tools like Verrou introduce runtime slowdowns of 10× to 1000× on models with only a few million parameters. This makes collecting sufficient statistical samples for large-scale models (e.g., Large Language Models) effectively infeasible, as analyses can take days or weeks even with parallelization. There is a critical need for a framework that integrates seamlessly with existing DL ecosystems (specifically PyTorch), minimizes computational overhead, and scales to models with hundreds of millions of parameters without compromising performance.

Methodology

The authors propose Fuzzy PyTorch, a framework that integrates stochastic arithmetic into PyTorch through a novel C++ library named PRISM (Probabilistic Rounding with Instruction Set Management).

Core Components

PRISM Library:
- Implementation: PRISM implements two rounding modes: Stochastic Rounding (SR) and a newly proposed Up-Down Rounding (UD).
- Optimization: The library leverages the Highway library to utilize vectorized CPU instructions (AVX2, AVX-512), minimizing computational overhead. It supports scalar and vector instructions via static and dynamic dispatch to ensure portability across architectures.
- Operations: It covers fundamental floating-point operations ( $+, -, \times, \div, \sqrt{},$ and Fused Multiply-Add (FMA)). The SR operators (except FMA) use rounding-mode-free algorithms, while the FMA implementation is inspired by Verrou and based on the ErrFmaNearest algorithm.
- Threading: PRISM assigns a separate random generator to each thread to prevent seed state sharing and correlations in perturbations, enabling optimal multithreaded performance without synchronization.
Up-Down Rounding (UD):
- UD is a novel mode where the result of an already rounded operation is randomly rounded up or down to the next or previous floating-point number with equal probability ( $1/2$ ).
- While UD does not preserve exact floating-point operations like SR, it is significantly faster at the individual operation level.
Integration with PyTorch and Verificarlo:
- Fuzzy PyTorch extends the Verificarlo compiler (LLVM-based) to use PRISM as a backend.
- The system replaces scalar and vectorized floating-point operations in PyTorch (version 2.2.1) with calls to the PRISM library via new LLVM instrumentation passes.
- To ensure compatibility, the authors modified the PyTorch codebase (removing a noexcept keyword in the Module class) and compiled open-source BLAS/LAPACK libraries (replacing proprietary Intel MKL) and the ONNX runtime with Verificarlo.
- Specific functions prone to erroneous outputs under perturbation (e.g., exponential, logarithmic in SLEEF, and torch.argmax under UD) were excluded from instrumentation to maintain reliability.

Key Contributions

The paper outlines three primary contributions:

PRISM Library: A new C++ library implementing fast probabilistic rounding methods (SR and UD) for systematic floating-point error analysis, optimized with vectorized CPU instructions.
Seamless PyTorch Integration: A framework that instruments deep learning operations transparently, allowing for efficient stochastic arithmetic evaluation without altering the underlying model architecture.
Comprehensive Evaluation: A comparative analysis against Verrou (a state-of-the-art tool) across diverse use cases, including MNIST digit classification, FastSurfer (whole brain MRI segmentation), and WavLM (Parkinson's classification from speech).

Results

The evaluation demonstrates that Fuzzy PyTorch achieves significant performance gains while maintaining comparable numerical variability characteristics to existing tools.

Runtime Efficiency:
- Fuzzy PyTorch achieves runtime reductions of 5× to 60× compared to Verrou across various deep learning models.
- In the NAS Parallel Benchmarks, PRISM UD consistently yielded the lowest overhead, while PRISM SR showed comparable or lower slowdowns than Verrou SR and CADNA.
- In DL tasks, speedups ranged from 7× to 49× for SR mode and 5× to 60× for UD mode. Notably, this performance was achieved even though Fuzzy PyTorch used standard Netlib BLAS/LAPACK, whereas Verrou relied on the highly optimized Intel MKL library.
- Performance varied by architecture: UD was faster for CNN-based models (MNIST, FastSurfer), while SR was faster for the transformer-based WavLM model, likely because SR bypasses exact operations, reducing relative overhead in complex self-attention layers.
Numerical Variability and Accuracy:
- Harmonic Series: PRISM SR exhibited variability comparable to MCA RR, Verrou SR, and the Fasi & Mikaitis library, converging to the expected binary64 reference value. PRISM UD showed higher variability (as expected for a mode that perturbs already rounded values) but converged toward binary32 results.
- Model Metrics: Across MNIST, FastSurfer, and WavLM, Fuzzy PyTorch maintained model performance comparable to Verrou.
  - MNIST: Variability was effectively confined to the loss function, with metrics retaining ~6.17 significant digits.
  - WavLM: Metrics showed high numerical stability (~6.17 significant digits), though analysis of pre-max probabilities revealed underlying instability masked by the final operation.
  - FastSurfer: Minimum Sørensen–Dice scores remained extremely high (~0.9985), confirming consistent segmentation accuracy despite introduced variability.
- Embeddings: Analysis of internal feature maps showed consistent variability patterns across tools, though PRISM UD introduced slightly higher uncertainty in WavLM embeddings.

Significance and Claims

The paper claims that Fuzzy PyTorch provides an efficient, scalable, and practical solution for assessing numerical variability in deep learning. Its primary significance lies in making large-scale, systematic studies of floating-point behavior feasible.

Scalability: By reducing runtime overhead by up to 60×, the framework enables researchers to run the multiple stochastic executions (typically 10+) required for stable statistical estimates on modern large-scale models, a task previously considered computationally prohibitive.
Reproducibility and Robustness: The tool allows researchers to quantify and manage floating-point uncertainty without compromising computational efficiency, directly supporting research on reproducibility and robustness in neural networks.
Practical Optimization: The authors highlight that efficient variability analysis can lead to practical improvements. For instance, the analysis of FastSurfer using Fuzzy PyTorch revealed regions of numerical instability in the background, which were subsequently exploited to implement "Conservative & Aggressive NaNs," yielding significant computational efficiency gains.
Future Outlook: While currently CPU-based, the authors anticipate that the findings will generalize to GPU architectures. They identify GPU support as a future direction, noting the technical challenges of fragmented tooling ecosystems but emphasizing the necessity of such extensions as hardware-supported rounding modes (e.g., in AWS Neuron, NVIDIA Blackwell) become more common.

In summary, Fuzzy PyTorch bridges the gap between rigorous numerical analysis and the practical demands of modern deep learning, enabling the community to move beyond theoretical concerns about floating-point precision to actionable, scalable evaluations of model reliability.