Generalized Lanczos method for systematic optimization of neural-network quantum states

This paper introduces the NQS Lanczos method, a systematic approach that combines supervised learning and variational Monte Carlo to optimize neural-network quantum states, effectively addressing underfitting and improving energy accuracy in highly frustrated quantum systems with linearly increasing computational cost.

The Gist

Imagine you are trying to find the lowest point in a vast, foggy, and incredibly complex mountain range. This mountain range represents a quantum system (like a collection of interacting atoms). The very bottom of the deepest valley is the ground state—the most stable, lowest-energy configuration of the system. Finding this spot is crucial for understanding how materials behave, but the map is so huge and the terrain so twisty that traditional methods often get lost or give up.

In recent years, scientists have started using Artificial Intelligence (AI) to help. They use a "neural network" (a type of AI brain) to guess what the shape of the valley floor looks like. This is called a Neural-Network Quantum State (NQS).

However, there's a problem: The AI is good at guessing, but it's not perfect. It's like a hiker who has a good map but keeps missing the exact lowest dip because the fog is too thick.

This paper introduces a new, systematic way to fix the AI's map. The authors call it the NQS Lanczos Method. Here is how it works, broken down into simple analogies:

1. The Problem: The "Exponential Wall"

Imagine trying to count every single grain of sand on a beach. As the beach gets bigger, the number of grains doesn't just grow; it explodes. In quantum physics, as you add more particles, the complexity grows so fast that even supercomputers can't calculate the exact answer. This is the "exponential wall."

2. The Old Way: The "Guess and Check" Loop

Previously, scientists used a method called Variational Monte Carlo (VMC). Think of this as the AI hiker walking around, looking for a lower spot, and adjusting their map slightly every time they find one. It works, but it can get stuck in a small dip that isn't the true bottom of the valley.

3. The New Method: The "Lanczos Ladder"

The authors combined the AI hiker with a mathematical tool called the Lanczos method. Imagine the Lanczos method as a way to build a ladder out of the fog.

• Step 1: The AI Hiker (Supervised Learning)
  The AI starts with a rough guess of the valley floor. The Lanczos method then asks: "If we take a step in the direction of steepest descent, where would we end up?"
  Instead of calculating this new position with impossible math (which would take forever), the AI is taught to recognize this new position. It's like showing the AI a photo of the "next step" and saying, "Draw this." The AI learns to mimic the shape of this new, slightly better state.

• Step 2: Building the Ladder
  The AI repeats this process. It learns to draw the first step, then the second step, then the third. Each step is a "Lanczos state." Now, instead of just one guess, the AI has a ladder of several guesses (states) that are all slightly better than the last.

• Step 3: The Superposition (Mixing the Ladder)
  Once the AI has built a ladder of these states, the method doesn't just pick the best one. It mathematically mixes them all together (like blending different colors of paint to get the perfect shade). This creates a "superposition state"—a new, highly refined map that is much closer to the true bottom of the valley than any single guess could be.

• Step 4: The Final Polish (VMC Optimization)
  Even after mixing the ladder, the map might still be a little blurry. The authors add a final step: they take this mixed map and run a final, intense round of "polishing" (VMC optimization). This smooths out the rough edges and ensures the AI has truly found the lowest point possible.

Why is this a Big Deal?

1. It's Efficient (The Linear Growth)
Previous attempts to use the Lanczos method with AI were like trying to carry a backpack that gets heavier every time you take a step. The more steps you took, the harder it became, until you couldn't move at all.
The new method is like a backpack with wheels. The effort required grows in a straight, manageable line. You can take many more steps (more "Lanczos steps") without the computer crashing. This allows them to solve much harder problems.

2. It Fixes the "Underfitting" Problem
Sometimes, the AI is too simple to learn the complex details of the mountain (this is called "underfitting"). The authors realized that even if the AI's drawing of the "next step" isn't perfect, mixing all the imperfect drawings together still creates a picture that is much better than the original. It's like taking a few slightly blurry photos and combining them to get a sharp image.

3. Real-World Results
They tested this on a famous, difficult puzzle in physics: the Heisenberg J1-J2 model (a grid of magnets that are very confused about which way to point). In the most confusing, "frustrated" areas of this puzzle, their method found a lower energy (a better solution) than almost any other method currently available, including those that use massive, complex neural networks.

The Bottom Line

The authors built a systematic training loop for AI in quantum physics. Instead of just letting the AI guess and hope, they give it a structured way to learn "next steps," build a ladder of improvements, mix them together, and polish the result.

It's like taking a rough sketch of a masterpiece, showing the artist how to improve it step-by-step, combining all the sketches, and then doing a final retouch. The result is a clearer, more accurate picture of the quantum world, achieved without needing a computer the size of a city.

Technical Summary

1. Problem Statement

The accurate determination of the ground state of quantum many-body systems is a central challenge in condensed matter physics, hindered by the "exponential wall" of the Hilbert space as particle numbers increase. While Neural-Network Quantum States (NQS) have emerged as powerful variational ansätze, often surpassing traditional methods like Variational Monte Carlo (VMC) and Density Matrix Renormalization Group (DMRG), they face specific limitations:

• Underfitting in Supervised Learning: When using supervised learning to represent complex states (like Lanczos states), neural networks often fail to capture the full wavefunction details, particularly in highly frustrated regimes or large lattice sizes, leading to significant underfitting.
• Computational Bottlenecks in Lanczos Methods: Previous attempts to combine the Lanczos method with NQS (e.g., using Restricted Boltzmann Machines) required calculating high-order Hamiltonian expectations $\langle H^{2i+1} \rangle$. This causes the computational cost to grow exponentially with the Lanczos step $i$, severely limiting the number of steps that can be performed.
• Sign Problem: Optimizing the sign structure of wavefunctions in frustrated systems remains difficult, often requiring fixed sign rules (like Marshall sign rule) which may not be optimal.

2. Methodology: The NQS Lanczos Method

The authors propose a systematic approach called the NQS Lanczos method, which integrates Supervised Learning, Variational Monte Carlo (VMC), and the Lanczos algorithm. The method consists of two main iterative components:

A. Supervised-Learning Lanczos (SLL) Algorithm

Instead of explicitly calculating high-order Hamiltonian powers, the SLL algorithm uses supervised learning to represent Lanczos states directly.

1. Initial State: Start with an initial NQS $|\psi_0\rangle$ (typically a CNN-based wavefunction).
2. Target Construction: For Lanczos step $i$, the target state $|\psi_i^{trg}\rangle$ is constructed using the recurrence relation of the Lanczos method:
   $$|\psi_{i+1}\rangle \propto H|\psi_i\rangle - a_i|\psi_i\rangle - b_i|\psi_{i-1}\rangle$$
   where coefficients $a_i$ and $b_i$ are calculated via Monte Carlo sampling.
3. Supervised Training: A neural network (specifically a real-valued Convolutional Neural Network with ResNet-v2 architecture, split into a Sign Network and an Amplitude Network) is trained to approximate $|\psi_i^{trg}\rangle$.
   • Sign Network: Treated as a binary classification task (positive/negative sign) using Cross-Entropy loss.
   • Amplitude Network: Trained to match the probability distribution $|\psi|^2$. The authors employ a Mean Squared Error (MSE) loss function based on samples from both the target distribution and the current network distribution to ensure robustness against underfitting.
4. Basis Construction: This process generates a set of basis NQSs $\{|\psi_0^{net}\rangle, \dots, |\psi_p^{net}\rangle\}$.
5. Diagonalization: A Hamiltonian matrix and an overlap matrix are constructed in this non-orthogonal basis. Diagonalizing the generalized eigenvalue problem yields a superposition state $|\Psi\rangle = \sum c_i |\psi_i^{net}\rangle$ and an improved energy estimate.

B. VMC Optimization Part

To address the underfitting inherent in the supervised learning step (where the network may not perfectly capture the target state), a VMC optimization step is applied to the superposition state $|\Psi\rangle$.

• The superposition coefficients and sign networks are fixed.
• The amplitude networks ($ANet_j$) of the constituent NQSs are optimized using the energy as the loss function.
• The Automatic Differentiable Monte Carlo (ADMC) method is used to calculate gradients efficiently.
• The resulting optimized state $|\tilde{\Psi}\rangle$ serves as the new initial state $|\psi_0\rangle$ for the next iteration of the NQS Lanczos loop.

3. Key Contributions

• Linear Scaling of Computational Cost: The primary theoretical contribution is the elimination of the $\langle H^{2i+1} \rangle$ calculation. By using supervised learning to represent Lanczos states, the computational cost scales linearly with the number of Lanczos steps $p$, enabling deeper Lanczos iterations than previously possible with NQS.
• Hybrid Optimization Strategy: The combination of SLL (to generate a diverse basis set) and VMC (to refine the amplitude and correct underfitting) creates a robust loop that systematically lowers the ground state energy.
• Advanced Network Architecture: The use of a dual-network CNN (Sign + Amplitude) with ResNet-v2 shortcuts and symmetry augmentation (translation, rotation, spin-flip) provides a powerful representation for 2D lattice systems.
• Loss Function Analysis: The paper systematically compares loss functions (MSE vs. Fidelity), demonstrating that MSE offers superior stability and performance for amplitude optimization in this context.

4. Results

The method was tested on the 2D Heisenberg $J_1-J_2$ model on square lattices with sizes $L=4, 6,$ and $10$, focusing on the highly frustrated region ($0.5 \lesssim J_2/J_1 \lesssim 0.6$).

• Small Systems ($L=4$): The method achieved near-exact diagonalization results, with relative errors dropping from $10^{-2}$ to $10^{-8}$ as Lanczos steps increased. The supervised learning perfectly captured the target states.
• Medium Systems ($L=6$):
  • The SLL algorithm significantly improved energies compared to the initial state.
  • The VMC optimization further reduced the error. For $J_2/J_1=0.5$, optimizing the amplitude network of the superposition state yielded energies closer to the exact diagonalization (ED) limit than the SLL step alone.
  • Sign Accuracy: The method improved the sign prediction accuracy from ~98% (initial) to 99.83% (superposition state), demonstrating its ability to refine the sign structure.
• Large Systems ($L=10$):
  • Despite the Hilbert space being too large for exact traversal ($2^{100}$), the method showed significant energy improvements.
  • While supervised learning showed signs of underfitting (loss convergence to $10^{-1}$ rather than $10^{-8}$), the VMC step effectively compensated for this, pushing the energy closer to the exact solution.
  • The results outperformed or were competitive with state-of-the-art methods like MinSR (Minimum-step Stochastic Reconfiguration) from recent literature.
• 1D Chain Validation: Tests on a 1D chain ($L=56$) confirmed that energy errors decrease monotonically as the number of Lanczos steps increases.

5. Significance

• Scalability: The linear computational cost scaling allows researchers to perform more Lanczos steps on limited hardware, systematically approaching the ground state without the exponential explosion of resources.
• Systematic Improvement: Unlike standard VMC which can get stuck in local minima, the NQS Lanczos method provides a systematic path to lower energies by expanding the variational space through the Krylov subspace construction.
• Robustness to Underfitting: The integration of the VMC step after the Lanczos diagonalization is crucial. It acts as a "correction" mechanism, ensuring that even if the supervised learning representation is imperfect, the final state is further optimized, making the method viable for large, frustrated systems where perfect supervised learning is impossible.
• Future Outlook: The authors suggest that increasing network size, exploring new architectures (e.g., Transformers), and designing better loss functions could further enhance the method's performance, potentially making it a standard tool for solving quantum many-body problems in condensed matter physics.