On the generalized eigenvalue problem in subspace-based excited state methods for quantum computers

This paper demonstrates that subspace-based excited state methods like QSE and qEOM suffer from severe instability due to the amplification of sampling errors by the condition number of the overlap matrix, whereas methods like q-sc-EOM that rely on standard eigenvalue equations offer a more robust and suitable alternative for quantum chemistry calculations on noisy quantum devices.

The Gist

Imagine you are trying to predict the future behavior of a complex machine, like a car engine, but you can only see it through a foggy, shaking window. You want to know not just how the engine runs normally (the ground state), but also how it behaves when you hit the gas hard or hit a bump (the excited states).

This paper is about two different ways quantum computers try to solve this "foggy window" problem. The authors found that one way is dangerously unstable when the window is shaky, while the other way is much more reliable.

Here is the breakdown using simple analogies:

The Two Competing Methods

Think of the quantum computer as a team of detectives trying to solve a mystery. They have a list of clues (data) they gather from the machine. To find the answer, they need to solve a math puzzle.

1. The "Old Way" (QSE - Quantum Subspace Expansion)

• The Analogy: Imagine the detectives are trying to solve a puzzle, but the pieces they are holding are slightly warped and sticky. To fit them together, they have to stretch and squish the pieces to make them align.
• The Problem: If the pieces are very warped (which happens when the math gets "ill-conditioned"), trying to stretch them to fit causes the whole puzzle to fall apart. A tiny mistake in how they hold a piece (statistical noise) gets blown up into a huge error in the final picture.
• The Result: Sometimes, the puzzle becomes impossible to solve. The detectives have to throw away the warped pieces (a technique called "thresholding") just to finish the puzzle. But the problem is, by throwing those pieces away, they lose some of the clues. They end up with a solution, but it's missing important parts of the story (missing excited states).

2. The "New Way" (q-sc-EOM - Quantum Self-Consistent Equation of Motion)

• The Analogy: This team of detectives uses a special set of puzzle pieces that are perfectly square and rigid. They don't need to stretch or squish anything to make them fit.
• The Advantage: Because the pieces are already perfect, they don't have to do that dangerous "stretching" step. Even if the window is shaking and the clues are a little fuzzy, the puzzle stays together.
• The Result: They get a complete picture with all the clues intact, and the answer remains stable even when the data is noisy.

The Core Problem: The "Foggy Window" (Noise)

Quantum computers today are "noisy." Every time they measure a piece of data, there is a little bit of static or "fog" (statistical sampling error).

• In the Old Way (QSE): The math requires them to divide by a number that represents how "aligned" their clues are. If that number is very small (which happens when the clues are messy), dividing by it acts like a magnifying glass for errors. A tiny bit of fog becomes a giant storm, ruining the calculation.
• In the New Way (q-sc-EOM): The math is set up so they never have to divide by that tricky number. The "magnifying glass" is removed. The fog stays small, and the answer remains clear.

What Happens When Things Get Bad?

The authors tested this with different scenarios:

1. Easy Cases (Low Noise/Good Alignment): Both methods work fine. It's like solving a puzzle on a calm day; both teams finish quickly.
2. Medium Cases (Getting Messy): The "Old Way" starts to panic. The answers jump around wildly, and the team needs to take way more measurements (shots) to get a decent answer. The "New Way" stays calm and accurate.
3. Hard Cases (Total Chaos): The "Old Way" hits a wall. The math breaks down completely (a singularity). To fix it, they have to use the "throw away pieces" trick. They get an answer, but it's incomplete—they missed some of the excited states. The "New Way" sails through without needing to throw anything away.

Why Does This Matter?

In chemistry, knowing all the possible states of a molecule is crucial. If you are designing a new drug or a solar cell, missing a specific "excited state" is like missing a crucial step in a recipe. You might think the cake will rise, but it won't.

The paper concludes that for the quantum computers we have today (which are noisy), the q-sc-EOM method is the better choice. It avoids the mathematical trap that causes errors to explode, ensuring that scientists get a complete and reliable picture of how molecules behave, without losing any critical information.

In short: The "Old Way" tries to force a square peg into a round hole and breaks the peg. The "New Way" uses a peg that fits perfectly, so the job gets done right, even when the workbench is shaking.

Technical Summary

Here is a detailed technical summary of the paper "On the generalized eigenvalue problem in subspace-based excited state methods for quantum computers."

1. Problem Statement

The paper addresses a critical stability issue in subspace-based excited state methods for quantum chemistry on near-term (pre-fault-tolerant) quantum computers.

• Context: Methods like Quantum Subspace Expansion (QSE) and Quantum Equation of Motion (qEOM) rely on solving a generalized eigenvalue equation ($HC = SCE$) on a classical computer, where matrix elements ($H$ and $S$) are measured on a quantum computer.
• The Core Issue: The overlap matrix ($S$) in these methods is often ill-conditioned. When statistical sampling errors (shot noise) inherent to quantum measurements are present, the condition number ($\kappa(S)$) of the overlap matrix dictates the stability of the solution.
• Consequence: As $\kappa(S)$ increases, the inversion of the overlap matrix (required to solve the generalized equation) amplifies sampling noise drastically. This leads to:
  1. Large errors in calculated excitation energies.
  2. Singularities where the equation becomes unsolvable without stabilization techniques.
  3. Loss of spectral completeness (missing excited states) when stabilization (thresholding) is applied.

2. Methodology

The authors perform a comparative analysis between two classes of subspace methods:

1. QSE (Generalized Eigenvalue Problem): Solves $HC = SCE$. The overlap matrix $S$ is non-trivial and must be inverted.
2. q-sc-EOM (Standard Eigenvalue Problem): Solves $HC = CE$. This method utilizes a self-consistent formulation with an orthonormal basis, resulting in an overlap matrix that is analytically the identity matrix ($S=I$).

Analytical Approach:

• The authors use matrix perturbation theory to derive error bounds for both standard and generalized eigenvalue equations.
• They demonstrate that for a generalized equation, the error in eigenvalues ($\delta\lambda$) scales inversely with the minimum eigenvalue of the overlap matrix ($\lambda_{min}(S)$):
  $$|\delta\lambda| \propto \frac{n(1+|\lambda|)}{\lambda_{min}(S)} \epsilon$$
  where $\epsilon$ represents shot noise. This proves that errors are amplified by the condition number of $S$.
• In contrast, for the standard equation (where $S=I$), the error scales linearly with noise without the amplification factor.

Numerical Approach:

• Systems: Simulations were performed on the H4 molecule (linear and square geometries) and planar NH3.
• Basis: STO-3G (H4) and STO-6G (NH3) with Jordan-Wigner mapping.
• Protocol:
  • Ground states were prepared using VQE (UCCSD ansatz) with exact (infinite shot) estimation to isolate excited-state method errors.
  • Excited states were calculated using QSE and q-sc-EOM with varying numbers of shots to introduce statistical noise.
  • Condition numbers of the overlap matrix were manipulated by changing molecular geometries (bond lengths).
  • Thresholding: For high condition numbers, the standard "thresholding" technique (discarding small eigenvalues of $S$) was applied to QSE to force solvability.

3. Key Contributions

1. Theoretical Quantification: The paper provides a rigorous derivation linking the condition number of the overlap matrix to the amplification of shot noise in generalized eigenvalue problems. It establishes that the required number of shots to achieve a target accuracy scales quadratically with the condition number ($\kappa(S)^2$).
2. Identification of Instability Regimes: The authors categorize the behavior of QSE into three regimes based on the condition number:
   • Low ($\kappa < 10^2$): QSE and q-sc-EOM perform comparably.
   • Medium ($10^2 < \kappa < 10^4$): QSE exhibits significant instability and variance compared to q-sc-EOM, even with moderate shot counts.
   • High ($\kappa > 10^4$): QSE becomes singular and unsolvable without thresholding.
3. Demonstration of Spectral Loss: The study shows that while thresholding stabilizes the QSE solution in high-condition regimes, it does so by truncating the subspace, leading to missing excited states (incomplete spectra).
4. Validation of q-sc-EOM: The authors demonstrate that q-sc-EOM, by avoiding the generalized eigenvalue problem (via $S=I$), remains stable and accurate across all condition numbers without requiring thresholding or losing spectral completeness.

4. Results

• Low Condition Number: For H4 with $\kappa(S) = 7.1$, both QSE and q-sc-EOM showed similar convergence and variance, confirming that overlap conditioning is not the dominant error source in well-conditioned cases.
• Medium Condition Number: For linear H4 with $\kappa(S) \approx 592.5$:
  • QSE: Showed a massive increase in error and variance. To achieve accuracy comparable to q-sc-EOM, QSE required approximately 10 times more shots ($10^5$vs$10^4$).
  • q-sc-EOM: Remained stable and accurate, unaffected by the ill-conditioning of the QSE overlap matrix.
  • Artificial Exactness Test: When the overlap matrix $S$ in QSE was artificially set to its exact (infinite shot) value, the errors dropped significantly, confirming that noise in $S$ and its inversion are the primary drivers of instability.
• High Condition Number: For linear H4 with $\kappa(S) \approx 2.05 \times 10^{12}$:
  • QSE: Could not be solved without thresholding. Applying thresholding ($5 \times 10^{-3}$) resolved the singularity but resulted in missing excited states (specifically degenerate states and states in specific energy ranges).
  • q-sc-EOM: Successfully computed the full set of excited states without thresholding.
• NH3 Case: Similar trends were observed in planar NH3, where increasing bond stretch led to higher condition numbers and QSE instability, while q-sc-EOM remained robust.

5. Significance and Conclusion

• Chemical Relevance: Accurate excited state calculations require a complete and precise spectrum. Missing states or high uncertainty can lead to incorrect interpretations of electronic structure, conical intersections, and reaction dynamics.
• Algorithmic Recommendation: The paper argues that q-sc-EOM is a superior candidate for excited-state quantum chemistry on noisy quantum hardware because it avoids the noise-amplifying step of inverting an ill-conditioned overlap matrix.
• Broader Impact: The findings extend beyond QSE and q-sc-EOM. Any quantum algorithm relying on generalized eigenvalue equations with measured matrix elements (e.g., quantum Krylov methods, non-orthogonal eigensolvers) faces similar stability risks.
• Resource Implications: The study highlights that simply increasing shot counts is not a scalable solution for ill-conditioned QSE problems due to the quadratic scaling of resource requirements. Structural changes to the algorithm (like the orthonormal basis in q-sc-EOM) are necessary to mitigate these fundamental errors.

In summary, the paper establishes that the generalized eigenvalue problem is a bottleneck for subspace-based excited state methods on quantum computers due to condition-number-driven error amplification, and proposes q-sc-EOM as a more stable, reliable alternative that preserves spectral completeness.