Tackling instabilities of quantum Krylov subspace methods: an analysis of the numerical and statistical errors

This paper analyzes the stability of quantum Krylov subspace methods and reveals that while they face numerical ill-conditioning in ideal settings, their performance in realistic noisy environments is primarily limited by statistical fluctuations, prompting the introduction of new filtering metrics to reliably assess solution quality without prior knowledge of the true spectrum.

The Gist

Imagine you are trying to find the lowest point in a vast, foggy mountain range. This "lowest point" represents the most stable energy state of a molecule, a crucial piece of information for designing new drugs or materials.

In the world of quantum computing, scientists use a clever mathematical trick called the Krylov subspace method to find this low point. Think of this method as sending out a team of scouts (called "references") to explore the terrain. The more scouts you send and the longer they walk (more "iterations"), the better your map of the valley becomes.

However, there's a catch. As the team gets bigger and the map gets more detailed, the data starts to get messy. The paper you shared investigates why this happens and how to fix it. Here is the breakdown in simple terms:

The Problem: The "Crowded Room" Effect

Imagine your team of scouts is trying to describe the terrain to you.

• The Ideal Scenario: In a perfect, noise-free world, if you send too many scouts, they start bumping into each other and repeating the same information. They become "linearly dependent." Mathematically, this creates a badly conditioned problem. It's like trying to solve a puzzle where half the pieces are identical copies of each other; the computer gets confused and can't find the unique solution.
• The Real-World Scenario: In actual quantum computers (which are currently noisy), the problem isn't just that the scouts are repeating themselves. It's that the scouts are shouting over each other due to static noise (called "shot noise"). The data they send back is fuzzy.

The Old Misconception

For a long time, scientists thought the main enemy was the "crowded room" (the mathematical instability). They tried to fix it by throwing away scouts who seemed to be repeating information (a technique called regularization).

The New Discovery: It's Actually the "Static"

This paper argues that in the real, noisy world, the "crowded room" isn't the biggest problem. The real villain is the statistical noise (the shouting/static).

• Even if you fix the "crowded room" issue, the noise in the data can still make the computer spit out garbage answers.
• The authors found that simply trying to clean up the math (regularization) helps, but it's not enough on its own.

The Solution: Two New "Lie Detectors"

The authors introduced two new tools to check if the computer's answer is trustworthy. They call these Imaginary Filters and Unitary Filters.

Think of these filters as a Lie Detector Test for the computer's answer:

1. The Unitary Filter (The "Perfect Circle" Test):

   • In the ideal world, the answers should form a perfect circle (mathematically, they should have a "norm" of 1).
   • If the answer is wobbly and the circle is squashed or stretched, the filter says, "Hey, this answer is unreliable! Throw it out."
   • Analogy: Imagine asking a group of people to draw a perfect circle. If someone draws a lopsided blob, you know they are either confused or the paper is shaking. You ignore their drawing.

2. The Imaginary Filter (The "Ghost" Test):

   • In the ideal world, energy numbers should be "real" numbers (like 5 or 10). They shouldn't have "imaginary" parts (like $5i$).
   • If the computer gives an answer with a "ghost" (an imaginary component), it's a sign that the calculation has gone off the rails due to errors.
   • Analogy: If you ask for the price of an apple and the cashier says "$5 and a ghost," you know something is wrong with the register. You don't pay; you ask for a new calculation.

The Big Takeaway

The paper concludes that:

1. Don't panic about the math getting messy: The "ill-conditioning" (the crowded room) isn't the end of the world.
2. Focus on the noise: The real issue is the static noise from the quantum hardware.
3. Use the filters: By using these new "Lie Detectors," scientists can automatically spot and discard bad answers without needing to know the correct answer beforehand.

In summary: Quantum computers are like noisy, chaotic construction sites. Instead of just trying to organize the workers better (regularization), this paper suggests installing a quality control camera (the filters) that instantly spots when a worker is building a wall that doesn't make sense, ensuring the final building (the energy calculation) is safe and accurate.

Technical Summary

1. Problem Statement

Quantum Krylov subspace methods (QKSM) are promising early fault-tolerant algorithms for estimating ground-state (GS) energies of quantum systems. However, they face a critical challenge: the ill-conditioning of the Generalized Eigenvalue Problem (GEVP).

• The Core Issue: As the Krylov subspace size increases, the basis vectors often become nearly linearly dependent. This leads to a high condition number ($\kappa(S)$) for the overlap matrix $S$, causing numerical instability during classical diagonalization.
• The Misconception: It is widely assumed that ill-conditioning is the primary barrier to accuracy in QKSM.
• The Reality in Noisy Settings: In realistic scenarios with finite sampling (shot noise), the problem is not just ill-conditioning but statistical fluctuations in the matrix elements of the Hamiltonian ($T$) and overlap ($S$) matrices. These fluctuations break the algebraic structure of the problem (e.g., Hermiticity or unitarity), leading to spurious solutions that cannot be reliably filtered by traditional metrics alone.

2. Methodology

The authors conducted a comprehensive numerical analysis using two strongly correlated molecular systems: triangular BeH$_2$ and rectangular H$_6$. They compared two variants of the Quantum Block Krylov Subspace algorithm:

1. QBKS-H: Uses the Hamiltonian $\hat{H}$ as the generator (Hermitian).
2. QBKS-U: Uses the real-time evolution operator $\hat{U}(t) = e^{-i\hat{H}t}$ (Unitary).

Key Experimental Variables:

• Subspace Expansion: Varied the number of Krylov iterations ($K$) and the number of initial reference states ($B$).
• Regularization: Tested various strategies to mitigate ill-conditioning, including:
  • Fixed singular value thresholding ($\sigma$).
  • "Elbow" detection (Kneed algorithm) on the singular value distribution.
  • Literature-based adaptive thresholds (dependent on noise level and subspace size).
• Noise Simulation: Simulated finite shot noise ($10^6$ shots per matrix element) to mimic realistic NISQ/early-FT hardware constraints.
• Filtering: Introduced new post-processing metrics to identify and discard unreliable solutions.

3. Key Contributions

A. Re-evaluation of Ill-Conditioning vs. Statistical Noise

The paper challenges the prevailing view that ill-conditioning is the dominant failure mode.

• Ideal Simulations: In noise-free environments, increasing the subspace size does lead to high condition numbers and instability.
• Noisy Simulations: In the presence of shot noise, the condition number $\kappa(S)$ is actually lowered because noise alleviates exact linear dependencies. However, the statistical noise in the matrix elements prevents accurate energy retrieval unless specific regularization is applied. The primary bottleneck shifts from "numerical instability" to "statistical error propagation."

B. New Reliability Metrics (Imaginary and Unitary Filters)

The authors propose two novel metrics to assess solution reliability without knowledge of the true eigenstate:

1. Imaginary Filter (for QBKS-H):
   • Theory: For Hermitian operators, eigenvalues should be real. If the overlap matrix $S$ loses positive definiteness due to noise or ill-conditioning, eigenvalues acquire imaginary parts.
   • Metric: $|\Im(\Lambda_i)|$. If the imaginary component exceeds a threshold (e.g., $1.6 \times 10^{-3}$ Hartree for chemical accuracy), the solution is discarded.
2. Unitary Filter (for QBKS-U):
   • Theory: For unitary evolution, eigenvalues must have a norm of exactly 1. Deviations from unity indicate errors.
   • Metric: $|1 - |\Lambda_i||$. Solutions where the norm deviates significantly from 1 are flagged as unreliable.

C. Optimization of Subspace Construction

• Single vs. Multiple References: While using multiple initial references ($B > 1$) reduces the condition number $\kappa(S)$, it increases the measurement cost (more distinct circuits). The authors found that a single reference with proper regularization and more iterations is more cost-effective for achieving chemical accuracy.
• Time-Step ($\tau$) Selection: Larger time steps generally yield more accurate energies for a fixed subspace size, but they require careful handling to avoid aliasing.

4. Results

• Regularization Efficacy:
  • Without regularization, unregularized solutions in noisy settings fail to reach chemical accuracy and are clearly flagged by the new filters (high imaginary parts or norm deviations).
  • Adaptive Thresholds: The literature method (b) proposed by Lee et al. (2024), which uses a noise-dependent threshold $\sigma \propto \sqrt{K \ln K} \times \text{Noise}$, yielded the highest accuracy for both algorithms.
  • Elbow Method: Discarding singular values based on the "elbow" of the distribution performed poorly, yielding higher errors and standard deviations.
• Filtering Performance:
  • Applying the Imaginary/Unitary filters successfully removed "spikes" in energy convergence caused by numerical instabilities, resulting in smooth convergence curves.
  • In noisy simulations, filtering alone was insufficient to reach chemical accuracy; it must be combined with regularization.
• Algorithm Comparison:
  • QBKS-U generally showed slightly better mean accuracy and lower standard deviations than QBKS-H under the tested regularization schemes, though its convergence was less smooth (step-wise).
  • Both algorithms successfully retrieved ground state energies within chemical accuracy ($\approx 1.6$ mHartree) for strongly correlated systems when using the proposed combination of regularization and filtering.

5. Significance and Conclusion

This work fundamentally shifts the understanding of stability in Quantum Krylov methods:

1. Condition Number is Misleading: $\kappa(S)$ is a poor indicator of solution reliability in noisy settings. A low condition number does not guarantee accuracy, nor does a high one guarantee failure in the presence of noise.
2. Practical Reliability Metrics: The proposed Imaginary and Unitary filters provide a practical, self-contained way to validate results on quantum hardware without needing a classical reference solution.
3. Path to Early-FT: The study demonstrates that with appropriate regularization (specifically adaptive thresholding) and post-processing filtering, Quantum Krylov methods can robustly solve strongly correlated problems even in the presence of significant sampling noise, making them viable candidates for early fault-tolerant quantum devices.

Conclusion: The paper concludes that while ill-conditioning is a theoretical concern, statistical fluctuations are the practical enemy. By combining adaptive regularization with the new reliability filters, quantum Krylov subspace methods can achieve chemically accurate results, overcoming the instability issues that have previously hindered their adoption.