Predicting quantum ground-state energy by data-driven Koopman analysis of variational parameter nonlinear dynamics

This paper proposes a data-driven method that applies Koopman analysis to the nonlinear dynamics of variational parameters to predict the ground-state energy of quantum Hamiltonians, offering a way to estimate energies even when the true ground state lies outside the variational manifold.

The Gist

Imagine you are trying to find the lowest point in a vast, foggy mountain range. In the world of quantum physics, this "lowest point" is called the ground state, and the height of the mountain represents the energy of a system. Finding this lowest point is crucial because it tells us how stable a material is, how it conducts electricity, or how it behaves at the atomic level.

Usually, scientists use a method called "Variational Methods" to find this low point. Think of this like trying to find the bottom of a valley by only walking on a specific, narrow dirt path you've laid out. If the true bottom of the valley happens to be in a swamp off your path, your method will fail. You'll find the lowest point on your path, but it won't be the true lowest point of the whole mountain.

This paper introduces a clever new trick to solve this problem using Machine Learning and a mathematical concept called Koopman Analysis. Here is how it works, broken down into simple analogies:

1. The Problem: The "Narrow Path"

In quantum physics, the "mountain" (the full quantum system) is so huge and complex that we can't map the whole thing. So, scientists create a simplified "map" (a variational wave function) that only covers a small, manageable area.

• The Issue: If the true answer lies outside this small area, the standard method gets stuck. It's like trying to find the ocean's lowest point while only walking on a single wooden plank.

2. The Solution: The "Magic Elevator" (Koopman Theory)

The authors use a mathematical tool called Koopman Analysis. Imagine your narrow dirt path is a winding, curvy road (non-linear). It's hard to predict where you'll end up just by looking at the curves.

Koopman theory is like a magic elevator that lifts you from that curvy road up into a giant, straight, infinite-dimensional sky.

• In the sky, the complex, curvy movement of the road becomes a simple, straight line (linear).
• Even though the road is curvy, the "elevator" allows us to analyze it using simple, straight-line math.

3. The Data-Driven Approach: Learning from "Good" Steps

The authors don't try to map the whole infinite sky. Instead, they take "snapshots" (data points) of the system.

• They look for moments where the system is behaving "well"—where the simplified path is very close to the true physics.
• They collect thousands of these snapshots.
• Using a machine learning technique called EDMD (Extended Dynamic Mode Decomposition), they ask the computer: "Based on these snapshots, what does the 'straight line' in the sky look like?"

4. The Result: Finding the Hidden Treasure

Once the computer reconstructs this "straight line" view (the Koopman generator), it can easily find the "lowest point."

• In this straight-line view, the lowest energy state shows up as a specific number (an eigenvalue).
• The Magic: Even if the true lowest point of the mountain was off the original narrow path, this method can still predict it! It's as if the computer looked at the few steps you took on the path, realized the shape of the whole mountain, and told you, "The real bottom is actually 50 feet to the left, in the swamp."

A Real-World Example from the Paper

The authors tested this on a simple model called the Transverse-Field Ising Model (think of it as a row of tiny magnets).

• They created a simplified model that couldn't perfectly describe the magnets' true behavior.
• They fed the computer data from this imperfect model.
• The Outcome: The computer successfully predicted the exact energy of the true ground state, even though the model they used was technically "wrong" or incomplete.

Why This Matters

This is a big deal because it combines the best of two worlds:

1. Physics: It respects the laws of quantum mechanics.
2. Machine Learning: It uses data to fix the mistakes of simplified models.

It's like having a GPS that knows the exact location of a destination, even if you are driving on a road that doesn't technically lead there. By analyzing the pattern of your driving, the GPS can tell you where the destination really is. This could help scientists design better materials, batteries, and quantum computers by finding energy states that were previously too hard to calculate.

Technical Summary

1. Problem Statement

The paper addresses the challenge of estimating the ground-state energy of quantum many-body systems, particularly in scenarios where the true ground state lies outside the chosen variational manifold.

• Context: Traditional variational methods (e.g., Variational Monte Carlo, Matrix Product States) optimize a trial wavefunction within a restricted parameter space. If the true ground state is not contained within this space, these methods yield an upper bound that may be inaccurate.
• Gap: There is a need for a method that can extrapolate or infer the true ground-state energy even when the variational ansatz is insufficient to capture the exact ground state.
• Core Idea: The author proposes leveraging Koopman theory to analyze the nonlinear dynamics of variational parameters during imaginary-time evolution. By treating the parameter evolution as a dynamical system, the method aims to recover the linear spectral properties (eigenvalues) of the original quantum Hamiltonian from data generated within the restricted variational space.

2. Methodology

The proposed framework connects the imaginary-time Schrödinger equation, variational parameter dynamics, and data-driven Koopman analysis.

A. Theoretical Formulation

1. Imaginary-Time Evolution: The system evolves according to $\frac{d}{d\tau}|\psi\rangle = -H|\psi\rangle$.
2. Variational Restriction: The state is restricted to a variational manifold $|\psi_\theta\rangle$ parameterized by a vector $\theta$.
3. Nonlinear Dynamics: The evolution of the parameters $\theta$ is generally nonlinear: $\dot{\theta} = f(\theta)$.
   • Ideally, if the manifold is closed, $f(\theta)$ is derived exactly.
   • In practice (restricted manifolds), the dynamics do not close. The author defines a residual $r(\theta)$ measuring the discrepancy between the true Hamiltonian action and the variational projection.
   • Sampling Strategy: Data points $(\theta, f)$ are collected only where the residual $r(\theta)$ is below a specific threshold. This ensures the sampled points lie in regions where the variational manifold approximates the true dynamics well.
4. Koopman Lifting:
   • Koopman theory lifts the finite-dimensional nonlinear dynamics of $\theta$ to an infinite-dimensional linear space of observables (functions of $\theta$).
   • The Koopman generator $\hat{L} = f(\theta) \cdot \nabla_\theta$ is a linear operator.
   • Key Insight: The eigenvalues of the Koopman generator correspond to the energy eigenvalues of the original quantum Hamiltonian ($-E_i$). Specifically, the ground-state energy corresponds to the leading eigenvalue of the generator.

B. Data-Driven Implementation (EDMD)

Since the functional form of $f(\theta)$ is often unknown or complex, the paper employs Extended Dynamic Mode Decomposition (EDMD):

1. Dictionary Construction: A set of basis functions (dictionary) $\Phi(\theta)$ is chosen (e.g., monomials of $\theta$).
2. Linear Regression: The relationship $\dot{\Phi} \approx L_N \Phi$ is solved via linear regression (using Moore-Penrose pseudoinverse) to estimate the finite-dimensional matrix approximation $L_N$ of the Koopman generator.
3. Eigenvalue Extraction: The eigenvalues of $-L_N$ are computed. The smallest eigenvalue provides the estimate for the ground-state energy.

C. Extension to Matrix Product States (MPS)

For large systems where full diagonalization is impossible, the method is adapted for Uniform Matrix Product States (MPS) on infinite chains:

• Utilizes the Time-Dependent Variational Principle (TDVP) framework.
• Efficiently computes the tangent space projection and the residual error without explicit diagonalization of the full Hamiltonian.
• Formulates the dynamics for the non-normalized state $|\psi_\theta\rangle = \theta_0 |\phi(A)\rangle$, allowing the extraction of energy densities and handling of norm conservation naturally.

3. Key Contributions

1. Novel Framework: Establishes a rigorous connection between the imaginary-time Schrödinger equation restricted to a variational manifold and the Koopman generator, proving that the ground-state energy is recoverable as the leading eigenvalue of this generator.
2. Data-Driven Extrapolation: Demonstrates that accurate ground-state energies can be predicted even when the variational space does not contain the true ground state, provided the sampling focuses on low-residual regions.
3. Integration with MPS/TDVP: Extends the method to infinite-system Matrix Product States, showing how to compute necessary quantities (dynamics, residuals) efficiently using established tensor network techniques.
4. Analytical Validation: Provides an analytical solution for a two-level system to verify the correspondence between Koopman eigenvalues and Hamiltonian eigenvalues, clarifying how to distinguish physical eigenvalues from spurious ones.

4. Results

The method was tested on the 4-site Transverse-Field Ising Model (a minimal example where the variational space is insufficient to close the dynamics):

• Setup: A translation-invariant variational state with two parameters ($\theta_1, \theta_2$) was used for a system with an 8-dimensional Hilbert space (even boson sector).
• Data Generation: 5,689 sample points were generated by optimizing $\theta$ until the residual was $< 10^{-3}$.
• EDMD Performance:
  • Using a monomial dictionary, the method estimated the ground-state energy.
  • Accuracy: With a dictionary degree of 3, the method reproduced the true ground-state energy ($0.45688$) with high accuracy.
  • Stability: Higher-degree dictionaries led to eigenvalue instability due to the non-normal nature of the Koopman matrix, highlighting the trade-off between model complexity and numerical stability.
  • Linear Baseline: Even a linear dictionary provided a reasonable prediction, suggesting the dynamics in the low-energy region are relatively weakly nonlinear.

5. Significance and Future Directions

• Complement to Variational Methods: This approach offers a way to "correct" or refine variational estimates. It is particularly valuable when the variational ansatz is known to be insufficient (e.g., missing long-range entanglement or specific correlations), as it can infer the true energy from the local dynamics of the parameters.
• Machine Learning Synergy: The method bridges physics-based variational principles with data-driven Koopman analysis, a growing field in dynamical systems. It suggests that ML techniques (like EDMD) can be used to extract physical constants from the trajectory of optimization parameters.
• Scalability: By integrating with TDVP and MPS, the framework is scalable to large quantum systems where exact diagonalization is intractable.
• Future Work: The author notes that future research should focus on:
  • Developing more sophisticated dictionary learning (e.g., using neural networks) to handle complex nonlinearities.
  • Addressing the bias introduced by non-uniform sampling in the variational space.
  • Applying the method to real-time dynamics and other variational Monte Carlo contexts.

In summary, Okuma presents a theoretically grounded and numerically validated method to predict quantum ground-state energies by treating variational parameter optimization as a dynamical system and applying Koopman spectral analysis to extract the underlying linear physics.