⚛️ quantum physics

Walshness: an intrinsic neural-network representability metric for quantum states

This paper introduces "Walshness," a rigorous complexity metric that quantifies the intrinsic neural-network representability of quantum states by proving that efficient representation is possible if and only if Walshness is bounded, and demonstrates how minimizing this metric to find optimal bases significantly improves the learnability of various physical quantum states.

Original authors: Nisarga Paul, Kehuang Chen, Jessica K. Jiang, Haimeng Zhao, Di Luo

Published 2026-10-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nisarga Paul, Kehuang Chen, Jessica K. Jiang, Haimeng Zhao, Di Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The universe at its smallest scale is governed by a set of rules that make the familiar world seem almost simple by comparison. In this quantum realm, particles do not just sit in one place or have one state; they exist in a vast, overlapping cloud of possibilities called a superposition. To describe even a modest collection of these particles, scientists must navigate a mathematical space so enormous that it grows exponentially with every new particle added. This "curse of dimensionality" has long been a barrier, making it nearly impossible for even the most powerful supercomputers to simulate complex quantum systems, such as the magnetic materials used in hard drives or the exotic states of matter found in superconductors.

To overcome this, researchers have turned to artificial intelligence, specifically a type of neural network designed to act as a flexible map of these quantum possibilities. These "neural quantum states" have shown great promise, learning to approximate the behavior of complex systems where traditional methods fail. However, a significant mystery remained: why do these networks sometimes learn a system's behavior in minutes, while other times they struggle for days or fail entirely? The answer seemed to depend heavily on how the system was viewed. Just as a photograph of a person looks different depending on the angle, the mathematical description of a quantum state changes drastically depending on the "basis," or the specific set of reference directions chosen to measure it. For years, scientists lacked a reliable way to predict which viewing angle would make a quantum state easy for a computer to learn and which would make it nearly impossible.

A new study introduces a solution to this problem, offering a new way to measure the inherent complexity of a quantum state. The researchers, led by Nisarga Paul and Di Luo, developed a metric they call "Walshness." This measure acts like a complexity score that tells us how difficult a quantum state is to represent using a neural network, but with a crucial twist: it accounts for the fact that the difficulty changes depending on how we choose to look at the system. The team proved mathematically that a quantum state is easy for a neural network to learn if and only if it has a low Walshness score in some specific viewing angle. Conversely, if a state has a high Walshness score in every possible angle, no neural network, no matter how large, can efficiently represent it. This discovery provides a rigorous, two-sided rule that connects the abstract difficulty of a quantum state to the practical ability of a machine to learn it.

The researchers did not just define this new metric; they also built a practical tool to find the best way to look at a quantum system. They showed that finding the angle that minimizes the Walshness score is mathematically equivalent to solving a classic problem in physics: finding the lowest energy state of a system of interacting magnets. By using this connection, they created an algorithm that can automatically rotate the quantum state into its "easiest" form. When they tested this method on real physical models, such as the transverse-field Ising model, which describes how magnetic spins align and flip, the results were striking. By training the neural network on the state after it had been rotated to minimize its Walshness, the researchers reduced the error in the simulation by orders of magnitude compared to using the standard, unrotated view. In some cases, the improvement was so significant that it turned a difficult, noisy learning process into a clean, rapid one.

This work also sheds light on long-standing puzzles in the field of frustrated magnets, where particles cannot all satisfy their magnetic preferences simultaneously. In these systems, a famous rule known as the Marshall sign rule has historically been used to simplify the math by flipping the signs of certain probabilities. The new study shows that the Walshness-minimizing algorithm naturally rediscovered this rule without being told about it. More importantly, it went further, finding even better ways to view the system in situations where the old rule was insufficient. This suggests that the new metric is not just a tool for improving current simulations but a deeper lens for understanding the structure of quantum matter itself.

The implications of this research extend beyond just making simulations faster. It offers a new way to diagnose the nature of quantum phases and transitions. The researchers found that the Walshness score behaves differently in various states of matter, such as in the topological phases of the toric code, a model used to study error-correcting quantum memories. In these complex states, the Walshness score revealed subtle features that correspond to known physical transitions, hinting that this metric could become a standard way to identify and classify exotic states of matter. By providing a clear, intrinsic measure of how "learnable" a quantum state is, the study bridges the gap between the theoretical limits of neural networks and the practical challenges of simulating the quantum world. It suggests that the key to unlocking the secrets of complex materials may not be building bigger computers, but simply learning to look at the quantum world from the right angle.

Title: Walshness: an intrinsic neural-network representability metric for quantum states

Problem Statement
Neural quantum states (NQS) have become a powerful tool for simulating quantum many-body systems, yet a fundamental theoretical gap remains: there is no quantifiable, intrinsic metric to predict when a specific quantum state can be efficiently represented by a neural network. While entanglement entropy dictates the efficiency of tensor networks, the complexity of NQS is complicated by the nonlinear parameterization of neural networks, the intricate sign structures of quantum wavefunctions, and a strong dependence on the choice of basis. Existing empirical studies suggest that a neural network capable of efficiently representing a state in one basis may fail in another, but a systematic theory linking basis choice to representability has been lacking.

Methodology
The authors introduce a new complexity metric called Walshness, derived from the Walsh-Hadamard transform (the discrete Fourier transform over $\mathbb{Z}_2^n$ ).

Definitions:
- Walsh Spectrum: For a state $|\psi\rangle = \sum_s \psi(s)|s\rangle$ , the Walsh-Hadamard transform yields coefficients $\hat{\psi}(r)$ . The Walsh spectrum $p_\psi(r) = |\hat{\psi}(r)|^2 / \sum |\hat{\psi}(r)|^2$ is treated as a classical probability distribution.
- Fixed-Basis Walshness ( $W^Z_\alpha$ ): Defined as $W^Z_\alpha[|\psi\rangle] = \frac{1}{\alpha} \log \mathbb{E}_{r \sim p_\psi}[e^{\alpha |r|_1}]$ , where $|r|_1$ is the Hamming weight. This metric quantifies the "oscillatory" nature of the wavefunction in a specific basis; high values indicate a distribution of Fourier modes concentrated at high Hamming weights.
- Intrinsic Walshness ( $W_\alpha$ ): To remove basis dependence, the authors define intrinsic Walshness as the minimum of $W^Z_\alpha$ over all single-site unitary transformations $U = \bigotimes U_i$ .
Optimization Algorithm:
The authors prove that minimizing intrinsic Walshness is equivalent to minimizing the energy of a classical spin model. Specifically, minimizing $W_\alpha$ corresponds to finding local axes $\{\hat{n}_j\}$ that minimize the expectation value $E[\{\hat{n}_j\}] = \langle \prod_j A_\alpha(\hat{n}_j) \rangle$ , where $A_\alpha$ is a specific operator dependent on $\alpha$ . This allows for the use of coordinate descent algorithms to find the optimal local basis.
Theoretical Proofs:
The paper establishes rigorous two-sided bounds connecting Walshness to the representability of Multilayer Perceptrons (MLPs) and Restricted Boltzmann Machines (RBMs):
- Forward Direction: A quantum state with low fixed-basis Walshness admits an efficient MLP representation (polynomial width, logarithmic depth) to any constant infidelity. Conversely, states with high Walshness require deep networks (depth $\Omega(\log n)$ or higher) to represent efficiently.
- Reverse Direction: A compact MLP representation implies the state must have low Walshness.
- RBM Extension: A similar "multiplicative Walshness" is defined for the logarithm of the wavefunction, establishing analogous bounds for RBMs.

Key Results

Theoretical Characterization: The authors provide a precise characterization of NQS learnability. They prove that a state is efficiently representable by a shallow neural network if and only if it has bounded Walshness in some basis.
Basis Optimization: The proposed algorithm successfully identifies local bases that minimize Walshness for various physical states.
Empirical Validation:
- Ferromagnets: The method correctly identifies that the $|+x\rangle$ state (constant wavefunction) has zero Walshness and is trivial to learn, while the $|-x\rangle$ state (parity function) has maximal Walshness and is exponentially hard to learn in the Z-basis.
- Transverse-Field Ising Model (TFIM): In the ferromagnetic phase ( $g < 1$ ), the Walshness-minimizing basis exhibits a period-two staggering that spontaneously breaks translation symmetry. Training NQS in this optimized basis reduces infidelity by orders of magnitude compared to the standard computational basis, particularly for larger system sizes ( $n=24, 26$ ).
- Frustrated Antiferromagnets: The method recovers the well-known Marshall sign rule for unfrustrated bipartite antiferromagnets and generalizes it to frustrated regimes (e.g., 2D XYZ antiferromagnets), finding basis rotations that lower Walshness beyond what the standard Marshall rule provides.
- Mixed-Field Toric Code: The approach demonstrates significant improvements in learnability for topologically ordered states when the reduction in Walshness is substantial, though advantages diminish near self-dual points where the reduction is minimal.

Significance and Claims
The paper claims to provide the first intrinsic complexity metric for quantum states that has provable, two-sided connections to neural-network representability, analogous to the role of entanglement entropy for tensor networks.

Diagnostic Tool: Walshness offers a new lens for diagnosing quantum states, distinct from entanglement and magic, potentially providing insights into quantum phases, criticality, and topological order.
Practical Utility: The metric serves as a practical tool for NQS practitioners. By minimizing Walshness to find an optimal local basis, one can significantly improve the trainability and accuracy of neural quantum states, often reducing infidelity by orders of magnitude.
Generalization of Heuristics: The framework recovers and generalizes established physical heuristics, such as the Marshall sign rule, suggesting a unified approach to handling sign problems in quantum Monte Carlo and NQS.

The authors remain modest regarding the scope, noting that their results are asymptotic in system size and primarily apply to underparameterized ansätze. They acknowledge that while the method improves optimization, it does not guarantee representability in overparameterized regimes and that the computational cost of finding the optimal basis scales exponentially with system size for exact methods, though the paper focuses on the theoretical existence and numerical demonstration of the principle.

More like this