⚛️ quantum physics

Interpretable Activation-Selection Neural Networks for Symbolic Regression of Parameter-Dependent Hamiltonian Eigenvalues

This paper introduces an interpretable activation-selection neural network that enforces dimensional homogeneity to derive explicit, compact symbolic expressions for parameter-dependent Hamiltonian eigenvalues, demonstrating its effectiveness in capturing perturbation-theory structures for spin-chain systems while noting that its accuracy matches, rather than exceeds, well-chosen fixed-basis models.

Original authors: Alexander Yu. Kharin, Kirill F. Sheberstov

Published 2026-08-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexander Yu. Kharin, Kirill F. Sheberstov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of physics, scientists often rely on powerful computers to solve complex problems that are too difficult to crack with a pen and paper alone. When studying tiny particles like atoms or molecules, these computers can calculate exact answers by crunching through massive amounts of numbers. However, these numerical answers often come as a wall of data that is hard to interpret. They tell us what happens, but they rarely explain why. To truly understand the rules governing nature, physicists prefer to find simple, compact mathematical formulas that describe how different factors influence one another. These formulas reveal hidden patterns, symmetries, and the fundamental laws that drive the universe. The challenge has always been finding a way to bridge the gap between the raw, precise numbers from a computer and the elegant, readable equations that scientists can actually use to gain insight.

A team of researchers has developed a new method to build this bridge, using a type of artificial intelligence designed to act like a translator between raw data and human-readable math. They focused on a specific problem involving the energy levels of tiny chains of particles, which are relevant to a technique called nuclear magnetic resonance used to study molecules. In these systems, the energy of the particles depends on how strongly they interact with their neighbors. While the exact energy values can be calculated precisely by a computer for any given set of interactions, finding a simple formula that describes these values across all possible conditions is notoriously difficult. The researchers created a special kind of neural network, a computer program inspired by the brain, that does not just predict numbers but actively searches for the best possible mathematical expression to describe them.

The core of their approach involves a library of basic mathematical building blocks, such as zero, a variable itself, and that variable squared. The network is trained to look at the input data and decide which of these building blocks to use at each step to construct the final answer. It is like giving a student a set of Lego bricks and asking them to build a structure that perfectly matches a complex shape, but with the added constraint that the student must also write down the exact instructions for how they put the bricks together. The researchers trained this system on data generated from computer simulations of three- and four-particle chains. They first converted the raw physical numbers into dimensionless ratios, a step that ensures the resulting formulas make sense regardless of the specific units of measurement used. This normalization helped the network focus on the underlying relationships rather than getting confused by the scale of the numbers.

When the researchers tested their system on the three-particle chain, the network successfully learned to produce compact formulas that matched the computer-generated data with high accuracy. These formulas were simple quadratic expressions, meaning they involved terms up to the second power, and they captured the general behavior of the energy levels across a wide range of conditions. The network managed to select the right combination of terms automatically, effectively rediscovering the structure of the solution without being told exactly what to look for. However, the researchers also found a crucial limitation. When they compared the network's results to a standard statistical method that simply fits a curve to the data using a pre-chosen set of mathematical terms, the standard method performed just as well, or even slightly better. This suggests that while the network is excellent at finding the right formula when the answer is a simple polynomial, it does not inherently possess a magical ability to outperform traditional methods if the correct type of formula is already known.

The study also explored more complex scenarios, such as the four-particle chain, where the interactions create a more intricate web of energy levels. Here, the network again succeeded in producing simple, readable formulas that described all eight different energy branches simultaneously. It correctly identified that the energy levels split into two distinct groups and captured the way they curved as the interactions changed. Yet, even in this more complex case, the network's performance was comparable to a straightforward curve-fitting approach. The researchers noted that the network struggled to reproduce certain sharp, non-smooth features that appear exactly where energy levels cross or touch, because the mathematical tools it was given were limited to smooth curves. This highlights that the network is only as good as the library of functions it is allowed to choose from; if the true answer requires a type of math that is not in the library, the network cannot invent it.

The most significant finding of this work is not that the new network is a superior calculator, but that it offers a different way to think about the problem. It demonstrates that a machine can be trained to select and combine mathematical functions in a way that results in a clear, explicit equation, rather than just a black box that spits out numbers. This is particularly valuable when scientists do not know in advance which mathematical form the answer will take. The network acts as a guide, suggesting which terms are important and which can be ignored, turning a sea of data into a concise statement of physical law. While it did not surpass the accuracy of traditional methods when the correct mathematical form was already known, it proved that it is possible to automate the discovery of these forms. The researchers conclude that this approach is a powerful tool for generating interpretable models, provided that the library of available mathematical functions is chosen carefully to match the physical reality of the problem being studied.

In the end, the work serves as a proof of concept for a new kind of scientific discovery tool. It shows that artificial intelligence can be directed to produce results that are not only accurate but also understandable to human scientists. By converting complex numerical simulations into simple algebraic expressions, the method helps researchers see the forest for the trees, revealing the fundamental scaling laws and symmetries that might otherwise remain hidden in the noise of raw data. The researchers emphasize that this is a step forward in making machine learning more transparent and useful for physics, offering a path toward automated discovery of the mathematical rules that govern the natural world.

Technical Summary: Interpretable Activation-Selection Neural Networks for Symbolic Regression of Parameter-Dependent Hamiltonian Eigenvalues

Problem Statement
While feedforward neural networks are universal approximators capable of modeling physical systems, they typically lack the ability to provide compact, explicit analytical expressions that reveal underlying physical laws, scaling behaviors, or symmetries. Symbolic regression aims to bridge this gap by finding mathematical expressions that balance accuracy with structural simplicity. However, deriving such expressions for parameter-dependent Hamiltonian eigenvalues—particularly for spin-chain systems relevant to zero-quantum nuclear magnetic resonance (NMR)—remains challenging. Numerical diagonalization yields exact data, but compact analytical approximations become increasingly difficult to derive as system size and parameter non-uniformity increase. Furthermore, standard regression approaches often struggle with dimensional consistency and the specific functional forms (e.g., polynomial vs. non-polynomial near degeneracies) required by physical constraints.

Methodology: Activation-Selection Networks (ASN)
The authors introduce the Activation-Selection Network (ASN), a differentiable architecture designed for symbolic regression. Unlike Kolmogorov-Arnold Networks (KANs) which learn univariate functions on edges, or Equation Learner (EQL) models which use sparsity regularization on operators, the ASN operates by selecting a single function from a predefined library for each input node.

Architecture: Each layer consists of an activation-selection sublayer followed by a linear-transformation sublayer. For an input $x_i$ , the network evaluates a set of $k$ candidate activation functions $\phi_j(x_i)$ .
Selection Mechanism: The network learns a vector of logits $\alpha_i$ for each input. During training, these are converted to coefficients $a_{ij}$ via a softmax transformation. To enforce sparsity (selecting one function), an $L_{0.5}$ -type regularization term is applied, which minimizes the sum of square roots of the coefficients, pushing them toward a vertex of the simplex (one coefficient $\approx 1$ , others $\approx 0$ ).
Sparsemax Transition: In the final stage of training, the softmax is replaced by sparsemax, which can assign exact zeros to unused functions, yielding a sparse analytical representation.
Function Library: The study utilizes the library $\{0, x, x^2\}$ . By stacking layers, the network can construct polynomials of arbitrary finite degree (theoretically up to $2^L$ for $L$ layers), allowing the composition of linear combinations and quadratic terms to generate higher-order polynomials.
Dimensional Normalization: A critical preprocessing step involves expressing Hamiltonian parameters and eigenvalues as dimensionless ratios (e.g., $r_i = \Delta J_i / J_{intra}$ ). This ensures dimensional homogeneity, reduces the number of independent variables, and prevents the learned expressions from depending on arbitrary energy units.

Benchmarks and Data
The framework was applied to effective spin-chain Hamiltonians describing coupled methylene proton pairs in zero-quantum NMR.

Three-Site Chain: Both uniform ( $\Delta J_1 = \Delta J_2$ ) and non-uniform cases were analyzed. The target was the four eigenenergy branches. The authors derived a two-parameter perturbative reference for the non-uniform case, noting that the exact splitting near degeneracy involves a radial term $\sqrt{\Delta J_1^2 + \Delta J_2^2}$ , which is non-polynomial.
Four-Site Chain: A uniform chain with eight eigenenergy branches was used to test scalability.
Data Generation: Numerical diagonalization provided exact reference data. For the three-site chain, 100,000 dimensionless parameter pairs were sampled. For the four-site chain, datasets were generated via direct diagonalization or interpolation of precomputed surfaces.

Key Results

Three-Site Chain: The ASN successfully extracted compact global quadratic approximations for the four eigenenergy surfaces. The resulting expressions captured the constant, linear, and quadratic structures expected from perturbation theory. Along the uniform-chain line, the ASN coefficients closely matched the qualitative structure of second-order perturbation theory (e.g., linear coefficients near $\pm 1/\sqrt{2}$ ).
Four-Site Chain: The ASN represented all eight eigenenergy branches simultaneously. The extracted expressions were reduced to quadratic polynomials with at most three terms. The network reproduced the two fourfold-degenerate manifolds at $\Delta J = 0$ and the qualitative perturbative structure for non-zero coupling.
Comparison with Baselines:
- When the appropriate polynomial basis was known in advance, fixed-basis least-squares regression (ordinary linear regression) slightly outperformed the ASN in terms of predictive accuracy (lower RMSE).
- For the three-site non-uniform case, adding a physics-informed radial feature ( $\rho = \sqrt{r_1^2 + r_2^2}$ ) to the least-squares model significantly improved local accuracy near the degeneracy, outperforming both the ASN and standard polynomial fits in that specific region.
- The ASN did not demonstrate an intrinsic accuracy advantage over a well-specified fixed-basis regression.

Significance and Claims
The paper positions the ASN not as a method that universally outperforms traditional regression, but as a differentiable framework for selecting and composing analytical building blocks when the appropriate functional form is unknown.

Primary Contribution: The ability to train a network to select a sparse combination of predefined functions and convert the result directly into an explicit symbolic expression without a separate post-hoc symbolic approximation step.
Modest Claims: The authors explicitly state that "adaptive activation selection does not provide an intrinsic accuracy advantage over a suitable predefined basis." The value of the ASN lies in its utility for problems where the correct basis is not known beforehand and multiple functional forms are plausible.
Limitations: The extracted expressions are approximations within the training domain and do not guarantee extrapolation. The method struggles with non-polynomial singularities (like the square-root dependence at degeneracies) unless such functions are explicitly included in the library. The study concludes that stronger evidence for adaptive function selection requires benchmarks with competing non-polynomial primitives and structured extrapolation tests.

In summary, the work establishes the ASN as a viable tool for deriving compact, interpretable symbolic representations from numerical data, particularly in scenarios where the underlying analytical structure is uncertain, while acknowledging that for problems with known polynomial structures, traditional least-squares methods remain highly competitive in accuracy.

More like this