Geometry-Induced Long-Range Correlations in Recurrent Neural Network Quantum States

This paper introduces dilated recurrent neural network wave functions as a computationally efficient alternative to transformers that successfully captures long-range correlations in quantum states by injecting an explicit geometric inductive bias, thereby overcoming the exponential decay limitations of standard RNNs in critical and highly entangled systems.

The Gist

Imagine you are trying to teach a computer to understand the complex, invisible rules that govern a crowd of tiny magnets (called "spins") dancing together in a quantum world. This is the job of Neural Quantum States (NQS). The computer acts like a detective, trying to guess the most likely arrangement of these magnets to find the system's "ground state" (its most stable, calmest energy level).

For a long time, scientists have used a specific type of detective called a Recurrent Neural Network (RNN). Think of an RNN as a person reading a story one word at a time. To understand the current word, they remember the previous words. It's great for short stories, but it has a major flaw: it has a short memory.

The Problem: The "Whispering Gallery" Effect

In a standard RNN, information travels like a whisper passed down a long line of people.

• Person 1 whispers to Person 2, who whispers to Person 3, and so on.
• If you are at the end of the line (Person 100), the message from Person 1 has been whispered so many times that it's barely audible. It gets "diluted" or lost.
• In physics terms, this means standard RNNs are terrible at understanding long-range correlations. They can easily see how two neighbors affect each other, but they struggle to see how two magnets far apart on opposite sides of the room are secretly connected.

When physicists tried to use these standard RNNs to simulate critical quantum systems (where things are on the edge of changing states), the computer failed. It predicted that the connection between distant magnets died away exponentially (like a whisper fading to silence), whereas in reality, the connection should fade much more slowly, following a "power law" (like a faint but persistent hum).

The Solution: The "Teleporting" Detective

The authors of this paper introduced a clever fix called Dilated RNNs.

Imagine you are still in that line of people passing a message, but you add a special rule: Every few people, you can skip ahead and whisper directly to someone far down the line.

• Layer 1: You whisper to the person next to you.
• Layer 2: You can now whisper to the person 2 spots away.
• Layer 3: You can whisper to the person 4 spots away.
• Layer 4: You can whisper to the person 8 spots away.

This is dilation. Instead of the message having to travel step-by-step (1 → 2 → 3 → 4...), it can "jump" (1 → 2 → 4 → 8...).

In the language of the paper, this changes the geometry of the network.

• Standard RNN: To get from the start to the end of a 100-person line, the message takes 100 steps.
• Dilated RNN: Thanks to the jumps, the message only takes about 7 steps (because $2^7 \approx 128$).

Why This Matters

By adding these "jumps" (dilated connections), the computer can finally "hear" the whispers from the far end of the line clearly.

1. The Math Magic: The authors proved mathematically that by changing the path the information takes, the way correlations fade changes from a rapid "exponential decay" (fading to zero quickly) to a slow "power-law decay" (staying relevant for a long time). This matches the real physics of quantum systems.
2. The Real-World Test: They tested this on two famous quantum puzzles:
   • The Ising Model: A chain of magnets at a critical tipping point. The standard RNN failed to see the long-range connections. The Dilated RNN saw them perfectly, matching the theoretical predictions.
   • The Cluster State: A highly complex, entangled state that previous RNNs couldn't solve at all. The Dilated RNN solved it smoothly and stably.

The Big Picture

Think of this like upgrading a city's transportation system.

• Standard RNNs are like a city with only local buses. If you want to get from the north side to the south side, you have to transfer buses 50 times. It takes forever, and you might get lost (the signal fades).
• Dilated RNNs are like adding express highways or teleporters that connect distant neighborhoods directly. You can get across the city in just a few hops.

The beauty of this paper is that it achieves this "express highway" effect without needing the massive computing power of other modern AI models (like Transformers). It keeps the speed of the old model but fixes the memory problem by simply changing the shape of the connections.

In short: The authors found a simple geometric trick—letting the AI "skip ahead" in its memory—that allows it to understand the deep, long-distance connections in quantum matter, solving problems that were previously impossible for this type of AI.

Technical Summary

1. Problem Statement

Neural Quantum States (NQS) based on autoregressive Recurrent Neural Networks (RNNs) have emerged as a powerful tool for simulating quantum many-body systems due to their efficient sampling capabilities (avoiding Markov-chain autocorrelation) and favorable linear computational scaling ($O(N)$). However, standard RNN architectures suffer from a fundamental limitation: they are inherently biased toward finite-length correlations.

• The Core Issue: Standard RNNs typically exhibit exponentially decaying correlations. This makes them unsuitable for accurately representing quantum ground states that possess long-range dependencies or power-law correlations, such as those found at critical points or in highly entangled states like the Cluster state.
• Current Alternatives: While Transformer-style self-attention mechanisms can capture long-range dependencies, they introduce substantial computational and memory overhead, scaling quadratically ($O(N^2)$) with system size, which limits their applicability to large-scale quantum simulations.

2. Methodology

The authors propose a novel architecture called Dilated RNN Wave Functions to address the correlation limitations of standard RNNs while maintaining computational efficiency.

• Architectural Innovation: The method introduces dilated connections into the recurrent units. Instead of only connecting to the immediate previous time step, recurrent units access information from distant sites through "dilated" connections.
  • The network consists of $L$ layers.
  • The dilation length for the $l$-th layer is defined as $s(l) = 2^{l-1}$.
  • For a system of size $N$, the depth is $L = \lceil \log_2 N \rceil$.
  • The hidden state at layer $l$ and site $n$ depends on the hidden state at site $n$ from the previous layer and the hidden state at site $n - 2^{l-1}$ from the same layer.
• Computational Scaling: This architecture retains a favorable forward-pass scaling of $O(N \log N)$, significantly better than the $O(N^2)$ scaling of Transformers, while enabling direct access to distant sites.
• Theoretical Analysis: The authors perform a linearized perturbation analysis on a simplified scalar RNN model.
  • Standard RNN: They demonstrate that the shortest path between two sites separated by distance $n$ scales linearly ($\ell_{min} \sim n$). Consequently, signal attenuation leads to exponential decay of correlations ($C_n \propto e^{-cn}$).
  • Dilated RNN: They show that the shortest path in a dilated RNN scales logarithmically ($\ell_{min} \sim \log n$). This geometric change reduces the multiplicative attenuation, theoretically inducing a power-law lower bound on correlations ($C_n = \Omega(n^{-\alpha})$).

3. Key Contributions

1. Introduction of Dilated RNNs for NQS: The paper proposes a simple geometric mechanism (dilation) to inject an explicit long-range inductive bias into autoregressive neural quantum states without the cost of attention mechanisms.
2. Analytical Proof of Correlation Geometry: The authors provide a rigorous analytical argument showing that dilation fundamentally alters the correlation geometry of the ansatz. They prove that while vanilla RNNs are generically associated with exponential decay, dilated RNNs admit regimes where connected two-point correlations are bounded from below by a power law.
3. Benchmarking on Critical Systems: The method is validated on two challenging quantum systems:
   • The 1D Transverse-Field Ising Model (TFIM) at the critical point.
   • The 1D Cluster State, a paradigmatic example of long-range conditional correlations known to be difficult for standard RNNs.

4. Results

The authors conducted numerical simulations using Variational Monte Carlo (VMC) with the Adam optimizer.

• 1D Transverse-Field Ising Model (TFIM):
  • Standard RNN: Single-layer and shallow RNNs failed to reproduce the expected physics, showing exponential decay of the connected two-point correlation function $C(r)$.
  • Dilated RNN: With $L \geq 4$ layers, the dilated RNN successfully reproduced the power-law decay ($C(r) \propto r^{-\eta}$) characteristic of the critical point.
  • Critical Exponent: The extracted critical exponent $\eta$ converged to the theoretical value of 0.25 (consistent with the (1+1)-dimensional Ising Conformal Field Theory), with a goodness-of-fit ($R^2$) close to unity.
• 1D Cluster State:
  • This state is known to be challenging for RNNs due to long-range conditional correlations. Previous studies reported significant performance degradation.
  • Performance: The 6-layer dilated RNN accurately approximated the ground state energy for $N=64$ with a relative error of $\approx 4 \times 10^{-5}$.
  • Stability: Unlike single-layer RNNs, which exhibited training instability and failed to converge, the dilated RNN converged smoothly and stably to the true ground state energy.

5. Significance

• Bridging the Gap: The work demonstrates that structural priors (geometry) can be as critical as representational expressivity in neural network design for physics. By incorporating dilation, the authors achieve long-range correlation modeling without the heavy computational cost of attention mechanisms.
• Scalability: The $O(N \log N)$ scaling makes this approach highly scalable for large quantum systems, offering a computationally cheaper alternative to Transformers for simulating critical quantum matter.
• Generalizability: The success on the Cluster state suggests that dilated RNNs can handle complex, non-stoquastic Hamiltonians and long-range conditional dependencies, potentially extending to 2D systems and other exotic lattices (e.g., Rydberg atom arrays).
• Theoretical Insight: The paper establishes a direct link between the architectural design (dilation depth) and the emergent correlation structure (power-law vs. exponential), providing a theoretical foundation for designing correlation-aware quantum neural networks.

In conclusion, the paper presents dilated RNNs as a principled, efficient, and effective solution for constructing autoregressive neural quantum states capable of capturing the long-range correlations essential for simulating critical and highly entangled quantum many-body systems.