A rigorous hybridization of variational quantum eigensolver and classical neural network

1. Problem Statement

The Variational Quantum Eigensolver (VQE) is a leading near-term algorithm for estimating ground-state energies in quantum chemistry and condensed matter physics. Its primary advantage is the Rayleigh–Ritz variational principle, which guarantees that the measured energy of any normalized trial state is an upper bound to the true ground-state energy ($E \ge E_{gs}$). This provides a built-in "sanity check" for optimization.

Recent approaches have attempted to enhance VQE by adding classical neural network post-processing layers. Specifically, Diagonal Non-Unitary Post-processing (DNP) (e.g., VQNHE) uses a neural network to reweight measurement outcomes (bit strings) to reshape the probability distribution, aiming to improve the expressiveness of shallow quantum circuits.

The Core Issue:
The authors identify that DNP introduces a fundamental statistical instability. DNP evaluates energy as a normalized ratio estimator:
$$E_f = \frac{\langle \psi | D_f^\dagger H D_f | \psi \rangle}{\langle \psi | D_f^\dagger D_f | \psi \rangle}$$
where $D_f$ is a diagonal, non-unitary operator defined by the neural network weights $f(x)$.

• Finite Sampling Bottleneck: In practice, the numerator and denominator are estimated from distinct, finite sets of measurement shots.
• Support Mismatch: If the neural network assigns high weights to bit strings that appear in the numerator (due to specific Pauli term measurements) but are not observed in the denominator (the ansatz sampling), the denominator fails to normalize these contributions.
• Consequence: The optimizer can exploit this "loophole" by driving weights on unobserved strings to extreme values, causing the empirical energy to drop below the true ground-state energy (sub-variational). This violates the physical principle of the algorithm and leads to non-physical results.

The paper asks: Can a data-driven neural post-processing layer be simultaneously (i) self-contained (no prior ground-state knowledge), (ii) scalable (polynomial resources), and (iii) variationally consistent (physically meaningful)?

2. Methodology and Theoretical Analysis

The authors rigorously analyze the limitations of DNP and propose a new architecture, U-VQNHE.

A. The "No-Go" Theorem for DNP

The authors prove that DNP cannot satisfy all three desiderata simultaneously under finite sampling:

1. Support Mismatch & Instability:

   • They define the "support inclusion" condition: The set of bit strings contributing to the numerator must be a subset of those observed in the denominator.
   • Proposition 1: If this condition is violated (which happens with high probability under polynomial sampling), the empirical objective is unbounded from below. The neural network can drive the energy to $-\infty$ by amplifying weights on unseen configurations.
   • Proposition 2 (Coupon Collector Problem): To guarantee support inclusion with high probability, the number of shots required scales exponentially with the number of qubits ($O(2^n)$).

2. Exponential Reweighting Range:

   • Even if one constrains the neural network output to a bounded range $[1/r, r]$ to prevent divergence, the authors prove that accurately reproducing the ground state requires the range parameter $r$ to grow exponentially with system size.
   • Theorem 1 (Constant-depth circuits): For constant-depth ansatzes, the overlap (Bhattacharyya coefficient) between the ansatz distribution and the ground state decays exponentially. Correcting this via reweighting requires an exponential dynamic range.
   • Theorem 2 (Linear-depth/2-design circuits): For linear-depth circuits (which form unitary 2-designs), the overlap with any fixed target state also decays exponentially ($O(2^{-n/2})$), again requiring exponential reweighting range.
   • Conclusion: Since the required measurement shots scale with the reweighting range (Eq. 15 in the paper), DNP requires exponential resources to achieve accuracy, rendering it non-scalable.

B. The Proposed Solution: U-VQNHE

To overcome these obstructions, the authors propose the Unitary Variational Quantum-Neural Hybrid Eigensolver (U-VQNHE).

• Core Principle: Enforce norm preservation by construction. Instead of learning non-unitary amplitudes (which require renormalization), the neural network learns a diagonal unitary transformation (a phase-only transformation).
• Mechanism:
  • The post-processing operator is $U_g = \sum_x e^{ig(x)} |x\rangle\langle x|$, where $g(x)$ is the real-valued output of the neural network.
  • Because $U_g$ is unitary, the state $|\psi_u\rangle = U_g |\psi\rangle$ remains normalized ($\langle \psi_u | \psi_u \rangle = 1$) automatically.
  • No Explicit Normalization: The energy estimator becomes a linear expectation value over the transformed state, eliminating the ratio estimator and the associated statistical bottleneck.
• Variational Safety: By Theorem 3, since the state remains normalized, the Rayleigh–Ritz bound is preserved by construction. The estimated energy can never fall below the true ground state.

3. Key Contributions

1. Unified Abstraction & Desiderata: Formulated a rigorous framework for diagonal neural post-processing and established three necessary criteria: self-contained training, polynomial resources, and variational consistency.
2. Proof of DNP Failure: Demonstrated mathematically that DNP-type methods inherently fail to meet these criteria simultaneously. They proved that finite sampling leads to support mismatch (causing sub-variational energies) and that accurate ground-state reconstruction requires exponential resources (shots or dynamic range).
3. U-VQNHE Algorithm: Introduced a normalization-free hybrid algorithm using diagonal unitary post-processing. This method retains the flexibility of a learnable layer while guaranteeing variational safety.
4. Numerical Validation: Demonstrated on Transverse-Field Ising Models (TFIM) that:
   • DNP (VQNHE) often produces non-physical energies below the ground state as system size increases or shots are limited.
   • U-VQNHE consistently stays within the physical bound and outperforms standard VQE in accuracy and robustness.

4. Results

• DNP Behavior: Simulations on 7–14 qubit TFIM models showed that under practical shot budgets (polynomial), DNP training trajectories frequently diverge or settle at energies significantly lower than the exact ground state. The neural network learns to assign extreme weights to unobserved bit strings to minimize the empirical loss, breaking physical consistency.
• U-VQNHE Performance:
  • U-VQNHE successfully improved upon standard VQE results without violating the variational bound.
  • It demonstrated robustness against shot noise, maintaining physical validity even as the number of qubits increased.
  • The method achieved better accuracy than both vanilla VQE and constrained DNP variants, proving that learning a phase landscape is sufficient to correct ansatz deficiencies without the instability of amplitude reweighting.

5. Significance

This work provides a critical theoretical correction to the field of hybrid quantum-classical algorithms.

• Architectural Insight: It highlights that "variational safety" is not just a property of the quantum circuit but of the entire hybrid loop. Introducing non-unitary, normalization-dependent post-processing fundamentally breaks the variational guarantee under realistic (finite-shot) conditions.
• Design Principle: The paper establishes "norm preservation by construction" as a mandatory design principle for scalable, physically meaningful hybrid algorithms.
• Future Direction: It shifts the focus from "amplitude reweighting" (which is statistically fragile) to "phase learning" (which is statistically robust), offering a viable path for enhancing VQE on NISQ hardware without requiring exponential resources.

In summary, the paper proves that the popular DNP approach is fundamentally flawed for scalable applications and provides a rigorous, unitary alternative (U-VQNHE) that resolves the trade-off between expressiveness and physical consistency.