⚛️ quantum physics

Neural Quantum States in Mixed Precision

This paper establishes both theoretical error bounds and empirical evidence demonstrating that mixed-precision arithmetic, particularly using half precision for sampling, can significantly enhance the scalability and energy efficiency of Neural Quantum States in Variational Monte Carlo simulations without compromising accuracy.

Original authors: Massimo Solinas, Agnes Valenti, Nawaf Bou-Rabee, Roeland Wiersema

Published 2026-01-29

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Massimo Solinas, Agnes Valenti, Nawaf Bou-Rabee, Roeland Wiersema

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the lowest point in a vast, foggy mountain range. This is what scientists do when they try to simulate complex quantum systems (like atoms interacting in a new material). The "lowest point" represents the most stable state of the system, known as the ground state.

To find this spot, they use a method called Variational Monte Carlo (VMC). Think of this as sending out thousands of hikers (samples) to explore the mountain. These hikers don't just wander randomly; they use a specific set of rules (a neural network) to guess where the low points might be, then take small steps to see if they can go lower. This process of "hiking" is called MCMC sampling.

The Problem: The Heavy Backpack

For decades, scientists have insisted that these hikers must carry double-precision backpacks (64-bit numbers). These backpacks are incredibly heavy and precise, ensuring that every step is calculated with perfect accuracy. However, carrying these heavy backpacks is slow and drains a lot of energy, especially when you are using modern super-fast computers (GPUs) that are actually designed to carry lighter loads.

The Solution: The Mixed-Precision Strategy

The authors of this paper asked a simple question: What if we let the hikers carry lighter backpacks for the walking part, but keep the heavy, precise ones for the important calculations?

They call this Mixed Precision.

The Walking (Sampling): They let the hikers use half-precision (very light, fast) or single-precision backpacks just to decide which direction to step.
The Planning (Training): They keep the heavy, double-precision backpacks for the actual math that updates the map and the neural network.

The Theory: Why It Doesn't Break

You might worry that using a lighter, less precise backpack would make the hikers get lost. The authors proved mathematically that this isn't the case, provided the hikers are moving fast enough.

They created a "safety net" theory:

The Noise is Small: The errors introduced by the light backpacks are like tiny, random bumps in the path.
The Hiking Speed Matters: If the hikers are moving quickly and exploring the whole mountain efficiently (a concept called "mixing"), these tiny bumps don't push them off course. The path they take still leads to the same lowest point.
The Result: As long as the "noise" from the light backpacks is small enough, the hikers will still find the exact same destination as if they were carrying the heavy backpacks the whole time.

The Experiment: Testing the Hikers

To prove this, the team ran simulations on a famous quantum model (the Transverse-Field Ising Model), which is like a grid of spinning magnets.

The Setup: They trained neural networks to find the ground state of these magnets.
The Test: They ran the "hiking" (sampling) part using different backpack weights: Double (heavy), Single, Half, and Brain (bf16).
The Outcome:
- Accuracy: The hikers using the light backpacks found the exact same lowest point as the ones with heavy backpacks. The final result was just as accurate.
- Speed: The hikers with light backpacks were up to 3.5 times faster.
- Why? Modern computer chips (GPUs) are built to process these lighter numbers much faster, just like a sports car handles a light load better than a heavy truck.

The Analogy of the "Bump"

Imagine you are walking on a tightrope.

Double Precision is like walking on a perfectly smooth, wide bridge.
Low Precision is like walking on a bridge with tiny pebbles (noise).
The paper shows that if you are walking fast and the pebbles are small, you won't fall off. You will still reach the other side safely. However, if the pebbles get too big (too much noise) or you walk too slowly, you might stumble. The authors calculated exactly how big the pebbles can be before you start to stumble.

The Bottom Line

This paper demonstrates that in the world of quantum simulation, you don't need to carry the heaviest possible backpack to get the job done. By switching to lighter, faster formats just for the "walking" part of the process, scientists can run simulations much faster and more efficiently without losing any accuracy. It's a way to get the same high-quality result with less energy and time.

Technical Summary: Neural Quantum States in Mixed Precision

Problem Statement
Scientific computing traditionally relies on double-precision (64-bit) arithmetic to ensure accuracy, particularly in simulations of real-world phenomena. However, the increasing availability of hardware accelerators like GPUs has made low-precision formats (e.g., single precision f32, half precision f16, bfloat16) attractive due to superior performance, reduced memory footprint, and improved energy efficiency. A significant bottleneck in modern scientific computing, specifically in Variational Monte Carlo (VMC) methods for solving quantum many-body systems, is the computational cost of generating sufficiently decorrelated Markov chain Monte Carlo (MCMC) samples. While low-precision arithmetic offers speedups, its impact on the accuracy of MCMC sampling—specifically regarding the introduction of bias and the potential destruction of ergodicity—remains under-explored. This work addresses the need to rigorously assess the applicability of mixed-precision arithmetic in MCMC-driven scientific applications, with a specific focus on Neural Quantum States (NQS).

Methodology
The authors approach the problem through a combination of theoretical derivation and empirical validation:

Theoretical Framework:
- The study models reduced precision as an additive perturbation $\delta(x)$ to the log-probability of the target distribution $\pi(x)$ , resulting in a perturbed distribution $\tilde{\pi}(x)$ .
- The authors derive analytical bounds on the error introduced by this perturbation in Metropolis-Hastings (MH) MCMC. They utilize the Total Variation (TV) distance to quantify the difference between the exact and perturbed distributions.
- By incorporating the mixing properties of the Markov chain (specifically using the Doeblin minorization condition to establish a contraction rate $r$ ), they derive a tighter bound on the bias than standard coarse bounds.
- Theorem III.3 provides a specific bound for local MH proposals with Gaussian noise, showing that the bias scales with the noise level $\sigma$ and the contraction rate $r$ . Crucially, if the chain mixes rapidly ( $r \ll 1$ ), the bias scales as $O(\sigma^2)$ rather than $O(\sigma)$ .
Empirical Validation:
- Toy Model: The authors first validate the theoretical bounds using a "noisy" Restricted Boltzmann Machine (RBM) where Gaussian noise is explicitly injected into the log-density to simulate precision errors.
- Mixed-Precision VMC Framework: They implement a practical mixed-precision scheme where the sampling component of VMC is executed in low precision (f16 or bf16), while the gradient computation and parameter updates (forward/backward passes) remain in double precision (f64). Parameters are updated in high precision and then downcast for the sampling step.
- Benchmarks: The method is tested on the Transverse-Field Ising Model (TFIM) in both antiferromagnetic and paramagnetic phases, as well as the Heisenberg model. Architectures tested include RBMs and Residual Convolutional Neural Networks (ResCNNs).

Key Contributions

Derivation of Error Bounds: The paper derives general analytical bounds on the sampling error introduced by low-precision arithmetic in discrete MH MCMC. It establishes a theoretical link between the noise level, the mixing time of the chain, and the resulting bias in the stationary distribution.
Mixed-Precision VMC Framework: The authors introduce a framework that performs MCMC sampling in low precision while maintaining high precision for the rest of the optimization loop. This effectively decouples the sampling bottleneck from the precision requirements of gradient updates.
Empirical Demonstration of Scalability: The work demonstrates that significant speedups (up to 3.5 $\times$ ) can be achieved in NQS optimization without degrading the training performance or the accuracy of the ground-state approximation.

Results

Theoretical Agreement: Empirical results from the noisy RBM toy model show excellent agreement with the derived theoretical bounds. The Monte Carlo sampling error in relative expectation values remains insensitive to perturbations until the noise parameter $\sigma$ becomes of order unity.
Accuracy Preservation: In full VMC optimizations for the 2D TFIM and Heisenberg models, the relative energy error with respect to a double-precision reference solution is nearly identical across f64, f32, f16, and bf16 data types.
Noise Analysis: The distribution of numerical errors ( $\delta$ ) in low-precision evaluations was found to be approximately Gaussian, validating the theoretical assumptions. The standard deviation of this error was consistently small ( $\sigma < 1$ ), ensuring that the MCMC chains could converge reliably.
Speedup Trends: Speedup factors increase systematically with both the number of samples and the number of model parameters. This is attributed to better GPU memory utilization and arithmetic unit saturation, respectively.
Gradient Noise: The study confirms that in f32 and f16 regimes, the stochastic Monte Carlo sampling noise dominates the low-precision numerical error, explaining why optimization dynamics remain stable.

Significance and Claims
The paper claims that mixed-precision arithmetic is a viable and effective strategy for accelerating MCMC sampling in scientific computing, specifically for Neural Quantum States. The authors assert that:

Substantial portions of the VMC algorithm, particularly the sampling of the quantum state, can be executed in half precision without loss of accuracy.
The derived theoretical framework provides a rigorous method to assess the applicability of mixed-precision in machine learning approaches relying on MCMC, extending beyond NQS to areas like Bayesian learning and energy-based models.
While mixed precision is standard in general machine learning, its application to MCMC-driven scientific computing has been under-explored. This work provides the theoretical justification and practical evidence needed to adopt these techniques for more scalable and energy-efficient simulations of quantum many-body systems.
The authors note that while sampling speedups are significant, further gains could potentially be realized by applying mixed precision to local energy computations and preconditioning steps, though this presents additional practical challenges.

The work concludes that mixed precision is a high-impact tool for the future of GPU-dependent scientific workflows, provided that the mixing properties of the underlying Markov chains are sufficiently robust to handle the introduced numerical noise.