🧬 biology

Numerical Optimization Strategies for the Variational Hamiltonian Ansatz in Noisy Quantum Environments

This study demonstrates that while gradient-based optimizers excel in noiseless simulations, population-based strategies like CMA-ES are significantly more robust against finite-shot sampling noise in Variational Hamiltonian Ansatz calculations, and that high-shot reevaluation can effectively mitigate sampling-induced fluctuations to recover accurate energy estimates.

Original authors: Silvie Illésová, Vojtěch Novák, Tomáš Bezděk, Clemens Sauer Possel, Martin Beseda

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Silvie Illésová, Vojtěch Novák, Tomáš Bezděk, Clemens Sauer Possel, Martin Beseda

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to bake the perfect chocolate cake, but you are doing it in a kitchen where the lights flicker wildly, the oven temperature jumps up and down, and every time you taste the batter, the spoon gives you a slightly different reading. This is the world of quantum computing today. Scientists are building machines that can solve problems impossible for normal computers, like simulating how molecules interact to create new medicines or materials. However, these machines are currently "noisy," meaning they make mistakes and give fuzzy answers. To get useful results, researchers use a clever trick called a Variational Quantum Eigensolver (VQE). Think of this as a game of "hot and cold" played with a recipe. You have a digital recipe (a quantum circuit) with many knobs you can turn (parameters). You want to turn the knobs until the cake (the molecule) tastes exactly right (has the lowest possible energy). But because the kitchen is noisy, every time you check the taste, the result is a bit jumbled.

To fix this jumbled taste, you need a classical optimizer—a smart computer program that acts like a sous-chef. Its job is to look at the noisy taste tests and decide which way to turn the knobs next to get closer to the perfect cake. For a long time, scientists assumed the best sous-chefs were the ones who could calculate the exact slope of the flavor curve (gradient-based methods), just like a hiker using a precise map to find the bottom of a valley. But this paper asks a crucial question: What happens when the map is covered in fog and the ground is shaking? Does the hiker with the precise map still win, or do they get lost? The authors of this study wanted to find out which "sous-chef" algorithm is the most robust when the quantum kitchen is messy and noisy, ensuring we can actually get good results from these fragile new machines.

The Great Optimizer Showdown in a Noisy Kitchen

In this study, the researchers set up a massive digital simulation to test eight different "sous-chef" algorithms. They didn't just test them on one recipe; they tried them on four different molecular "cakes": a simple hydrogen molecule ( $H_2$ ), a chain of four hydrogens ( $H_4$ ), and lithium hydride ($LiH$) in both its full and simplified forms. They used a specific, efficient recipe design called the truncated Variational Hamiltonian Ansatz (tVHA), which is like a streamlined version of a complex recipe that skips unnecessary steps to save time and reduce errors.

The team ran 1,280 independent simulations. They tested each algorithm under two conditions: a perfect, noise-free world (like a kitchen with perfect lighting and steady ovens) and a realistic, noisy world where they had to simulate "finite-shot sampling." In quantum terms, "finite-shot sampling" means you can't measure the energy perfectly; you have to take a limited number of samples (like tasting the batter 6,144 times) and average them out. This averaging introduces statistical noise, making the energy readings wobble.

The Plot Twist: The Map Becomes a Lie

The results were surprising and flipped the script on what scientists expected. In the perfect, noise-free simulations, the gradient-based methods (like BFGS and SLSQP) were the clear winners. They were fast, precise, and found the lowest energy levels with machine-like accuracy, reaching errors as small as $10^{-14}$ Hartree (a unit of energy). They were like hikers with a perfect GPS, zooming straight to the bottom of the valley.

However, once the researchers introduced sampling noise (the wobbly kitchen), the rankings completely reversed. The gradient-based methods, which relied on calculating precise slopes, got confused. The noise made the "slope" look like it was going the wrong way, or the ground looked flat when it wasn't.

BFGS, the champion in the clean world, became one of the worst performers in the noisy world, with errors jumping up to nearly $4 \times 10^{-3}$ Hartree.
SLSQP essentially gave up, failing to converge reliably even for the smallest molecule, $H_2$ .
The noise was so deceptive that it sometimes made the algorithms think they had found a cake better than the perfect one (violating the "variational principle," which says you can't get a better energy than the true ground state). This was just a statistical fluke, a lucky guess caused by the noise, not a real improvement.

The New Heroes: The Explorers

In the noisy environment, the winners were the population-based and stochastic methods, specifically CMA-ES, PSO (Particle Swarm Optimization), and SPSA.

CMA-ES (Covariance Matrix Adaptation Evolution Strategy) emerged as the most robust. Instead of trying to follow a single, precise path, it acted like a swarm of explorers. It kept a whole population of candidate solutions, learning from their collective experience. Even when the noise made individual paths look terrible, the group average smoothed out the errors.
For the $H_2$ molecule, CMA-ES achieved a corrected error below $5 \times 10^{-4}$ Hartree.
For the more complex $LiH$ molecule, it kept the error below $10^{-2}$ Hartree, far outperforming the gradient-based methods which struggled to stay below $2 \times 10^{-2}$ Hartree.

The study found that while gradient-based methods are great for small, clean problems, they crumble when the landscape gets rugged and noisy. In contrast, population-based approaches like CMA-ES are like a team of explorers who can navigate a foggy mountain range because they don't rely on a single, shaky compass; they use the wisdom of the crowd to find the path.

The "High-Shot" Reality Check

One of the most interesting findings was about those "magic" results where the algorithms seemed to find an energy lower than the true ground state. The paper explains that this is an illusion caused by noise. When the noise is high, random fluctuations can make a bad guess look like a great one.

To fix this, the researchers used a trick called high-shot reevaluation. After an optimizer found a "good" solution, they re-measured it with a massive number of samples ( $10^5$ shots) to get a super-precise reading.

This revealed that many of the "improvements" were just noise artifacts.
However, the population-based methods (like CMA-ES) were better at finding the true best solution even before this reevaluation, because their averaging mechanism naturally smoothed out the noise.
The study suggests that in noisy quantum environments, the goal shifts from just "finding the minimum" to "estimating the energy reliably." The best strategy is to use a robust optimizer like CMA-ES and then spend extra time (more shots) verifying the final answer, rather than trying to force a precise gradient method to work in the fog.

The Takeaway for the Future

The paper concludes with a clear guide for anyone trying to use these noisy quantum computers. If you are working on a small, simple problem with very little noise, you can use the fast, gradient-based methods. But as soon as you move to larger molecules or realistic noisy hardware, you should switch to population-based optimizers like CMA-ES, PSO, or SPSA.

The authors also noted that starting with a "chemically motivated" guess (using a Hartree-Fock initialization) helps in small systems, but as the problem gets bigger, random starting points sometimes work just as well because the search space is so vast that a little randomness helps explore more ground.

Ultimately, this research suggests that to get the most out of today's noisy quantum machines, we need to stop trying to force them to be precise like classical computers and start using algorithms that are designed to be robust, exploratory, and good at averaging out the chaos. The "perfect" map is useless in a storm; you need a team of explorers who can find their way together.

Technical Summary: Numerical Optimization Strategies for the Variational Hamiltonian Ansatz in Noisy Quantum Environments

Problem Statement
Variational Quantum Eigensolvers (VQEs) are promising for near-term quantum chemistry but face significant performance degradation due to sampling noise inherent in finite-shot measurements on Noisy Intermediate-Scale Quantum (NISQ) devices. While classical optimizers are widely used, their robustness under realistic finite-shot conditions—specifically for Hamiltonian-informed ansätze like the Variational Hamiltonian Ansatz (VHA)—remains insufficiently understood. Sampling noise distorts the optimization landscape, creates false minima, induces erratic convergence, and can lead to apparent violations of the variational principle (energy estimates below the true ground state). This study addresses the critical need to identify which classical optimization strategies remain reliable when gradients are obscured by statistical fluctuations.

Methodology
The authors conducted a comprehensive benchmarking study involving 1,280 independent optimization runs across four molecular systems: H₂, H₄ (chain), LiH (full space), and LiH (active space). The simulations utilized the truncated Variational Hamiltonian Ansatz (tVHA), a chemically motivated framework that reduces circuit complexity by truncating non-Coulomb two-body interaction terms while preserving particle-number and spin symmetries.

The study evaluated eight distinct classical optimization algorithms:

Gradient-based: Gradient Descent (GD), Broyden-Fletcher-Goldfarb-Shanno (BFGS), Sequential Least Squares Programming (SLSQP).
Derivative-free/Gradient-free: Simultaneous Perturbation Stochastic Approximation (SPSA), Constrained Optimization By Linear Approximations (COBYLA), Nelder-Mead.
Population-based: Covariance Matrix Adaptation Evolution Strategy (CMA-ES), Particle Swarm Optimization (PSO).

Simulations were performed using a Python stack (Qiskit, PySCF) on the Barbora supercomputer under two conditions:

Exact Statevector: Noise-free reference simulations.
Finite-Shot Sampling: Noisy simulations with a fixed budget of 6,144 shots per energy evaluation.

The study compared two initialization strategies: Hartree-Fock (HF) initialization (chemically motivated) and random initialization. A key methodological component was the "high-shot reevaluation" protocol, where candidate solutions found during noisy optimization were re-evaluated using $10^5$ shots to distinguish genuine convergence from statistical artifacts.

Key Contributions and Results

Reversal of Optimizer Rankings: The study demonstrates that finite-shot sampling noise fundamentally alters optimizer performance hierarchies. In noiseless statevector simulations, gradient-based methods (particularly BFGS) achieve near-machine precision and are the fastest. However, under sampling noise, these methods degrade significantly. BFGS and SLSQP become unreliable, with SLSQP frequently diverging and BFGS producing errors orders of magnitude higher than in noiseless cases.
Robustness of Population-Based Methods: In contrast to gradient-based approaches, population-based and stochastic methods (CMA-ES, PSO, SPSA) maintain robustness under noise. CMA-ES, in particular, consistently achieved the lowest corrected mean errors across all systems, reaching sub-millihartree accuracy for H₂ ( $<5\times10^{-4}$ Ha) and LiH active space ( $<10^{-3}$ Ha), and errors below $10^{-2}$ Ha for LiH full space.
Impact of Initialization: Hartree-Fock initialization generally stabilizes optimization trajectories and reduces error variance, particularly for smaller systems like H₂. However, as system complexity increases (e.g., LiH full space), the advantage of HF initialization diminishes, and population-based methods with random initialization sometimes outperform HF-initialized runs by exploring broader search spaces.
Variational Principle Violations and Noise Floor: The authors observed that finite-shot sampling frequently produces energy estimates below the true ground state, violating the variational principle. High-shot reevaluation revealed that these "improvements" are statistical artifacts (noise-driven overfitting) rather than genuine physical convergence. The study highlights that in noisy regimes, the primary challenge shifts from finding the minimum to obtaining statistically reliable energy estimates.
Active Space Reduction: Reducing the active space (e.g., LiH active space vs. full space) significantly improves optimization difficulty and noise robustness, allowing several optimizers to recover sub-millihartree accuracy even under sampling noise.

Significance and Claims
The paper claims to provide practical guidelines for optimizer selection in noisy variational quantum chemistry. It establishes that:

Context-Dependent Selection: Gradient-based optimizers are optimal only for small, low-noise problems. For larger systems or realistic noisy environments, stochastic and population-based optimizers (CMA-ES, PSO, SPSA) are essential for stable convergence.
Necessity of Reevaluation: High-shot reevaluation is critical to validate results, as low-shot optimizations often yield misleadingly low energies due to statistical fluctuations.
Nature of Noise: The study clarifies that sampling noise does not merely add error but actively reshapes the optimization landscape, creating "glassy" topographies where deterministic methods fail. It argues that the apparent violation of the variational bound is a statistical phenomenon of finite sampling rather than a physical improvement.

The authors conclude that while hardware noise (gate infidelity, decoherence) is a separate concern, sampling noise alone is sufficient to degrade deterministic optimizers and necessitate a shift toward noise-resilient, population-based strategies for practical NISQ applications. Future work is suggested to explore hybrid strategies and adaptive shot allocation, but the current study focuses strictly on the interplay between algorithm design and sampling noise.

The Great Optimizer Showdown in a Noisy Kitchen

The Plot Twist: The Map Becomes a Lie

The New Heroes: The Explorers

The "High-Shot" Reality Check

The Takeaway for the Future

More like this