⚛️ high-energy theory

Machine learning modularity

This paper introduces a transformer-based machine learning framework that achieves over 99% accuracy in simplifying complex expressions involving elliptic Gamma functions by learning to apply modular transformations, thereby offering a novel automated tool for symbolic computations in quantum field and string theories.

Original authors: Yi Fan, Vishnu Jejjala, Yang Lei

Published 2026-10-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yi Fan, Vishnu Jejjala, Yang Lei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of theoretical physics, where scientists try to understand the fundamental building blocks of the universe, there exists a powerful mathematical tool known as the partition function. Think of this function as a master ledger that counts every possible way a system of particles can arrange itself, from the tiniest vibrations of a string to the swirling heat of a black hole. For decades, physicists have relied on these functions to uncover deep secrets about the nature of reality, particularly how the laws of physics remain consistent even when the shape of space and time changes in specific, rhythmic ways. These rhythmic changes are called modular transformations, and they act like a set of hidden rules that allow different mathematical descriptions to represent the same physical truth. However, as the systems being studied become more complex—moving from simple two-dimensional surfaces to intricate, higher-dimensional structures—the mathematical expressions describing them grow wildly complicated. They become tangled webs of special functions that are incredibly difficult to untangle by hand, often obscuring the simple, elegant patterns that lie beneath the surface.

A team of researchers has now turned to artificial intelligence to solve this problem of mathematical clutter. By training a sophisticated computer model on the rules of these modular transformations, they have created a system capable of automatically simplifying these complex expressions. The researchers focused on two specific types of mathematical objects that appear frequently in theories describing supersymmetric quantum fields and string theory: the q-theta function and the elliptic Gamma function. These functions are the building blocks of the partition functions mentioned earlier, and they possess a unique property: when you apply certain transformations to them, they change in a predictable way, often reducing a messy product of many terms into a much simpler form. The challenge for human mathematicians is that these simplifications require navigating a labyrinth of algebraic identities, and the expressions can become so scrambled that the underlying simplicity is lost. The team's goal was to see if a machine learning model could learn these rules well enough to take a scrambled expression and instantly recognize its simplest, most canonical form.

To teach the computer, the researchers first had to generate a massive library of examples. They started with simple, clean expressions and then deliberately "scrambled" them by applying a sequence of known mathematical transformations. This process created millions of pairs of data, where each pair consisted of a complicated, jumbled expression and its corresponding simplified version. The computer model, based on a modern architecture designed for understanding sequences of language, was then trained to look at the jumbled input and predict the clean output. The researchers tested the model's ability to learn these rules under different conditions, including how the data was sampled and how deeply the expressions were scrambled. They found that the model could learn to identify the correct simplification with remarkable precision, achieving an accuracy of nearly ninety-nine percent on standard tests. More impressively, the model did not just memorize the examples it saw during training; it demonstrated a robust ability to generalize. When presented with expressions that were scrambled much more deeply than anything it had encountered before, the model still managed to find the correct simplified form with high accuracy, suggesting it had truly learned the underlying logic of the transformations rather than just recalling specific answers.

The success of this approach extends beyond the specific functions tested. The researchers showed that the model could handle not only the simpler q-theta functions but also the more complex elliptic Gamma functions, which involve additional layers of mathematical structure. In these more difficult cases, the model achieved an accuracy of over ninety-three percent even when tested on completely new, more complex scenarios. This is a significant step forward because it demonstrates that machine learning can be used to automate symbolic manipulation in areas of physics where human calculation is often too slow or prone to error. The researchers also clarified that while the model identifies the simplified structure of the expression, the specific numerical "phase" or offset that accompanies the transformation can be calculated separately using established mathematical algorithms. This division of labor allows the artificial intelligence to focus on the structural complexity of the problem, which is often the hardest part for humans to manage.

The implications of this work are practical and immediate for the field of theoretical physics. By automating the simplification of these expressions, scientists can more easily analyze the properties of black holes and the microscopic states of quantum systems. The ability to quickly reduce a tangled mathematical expression to its core form allows researchers to see the physical meaning that was previously hidden. The researchers noted that this framework could be particularly useful for studying partition functions on complex geometric shapes, such as lens spaces, where the mathematical identities are intricate and difficult to navigate manually. While the current work focuses on specific types of functions, the success suggests a path toward handling even more general families of mathematical objects that appear in the study of the universe. The study does not claim to have solved all problems in this area, but it provides a powerful new tool that proves machines can learn the deep, structural rules of advanced mathematics, offering a new way to explore the fundamental symmetries of nature.

Technical Summary: Machine Learning Modularity

Problem Statement

The paper addresses the challenge of simplifying complex symbolic expressions involving special functions that arise in supersymmetric quantum field theory (QFT) and string theory, specifically the $q$ - $\theta$ function and the elliptic Gamma function. These functions serve as partition functions for various supersymmetric theories (e.g., on $S^3 \times S^1$ ) and possess rich modular properties under $SL(r, \mathbb{Z})$ transformations.

While the modular identities (such as $SL(2, \mathbb{Z})$ for $\theta$ and $SL(3, \mathbb{Z})$ for $\Gamma$ ) are known, applying them to "heavily scrambled" expressions—those resulting from repeated modular transformations, duplications, and shifts—is computationally difficult. The goal is to automatically reduce such scrambled expressions to their canonical forms (minimizing the number of function factors) and identify the underlying modular transformations. Previous machine learning work had successfully simplified polylogarithm expressions, but extending this to modular functions requires handling the specific algebraic structures and phase factors (modular cocycles) associated with elliptic functions.

Methodology

1. Architecture and Training Framework

The authors employ a sequence-to-sequence Transformer architecture, specifically the T5 family (Flan-T5-small for $q$ - $\theta$ and Flan-T5-base for elliptic Gamma functions), implemented via the Hugging Face library.

Task Formulation: The simplification is framed as a supervised sequence-to-sequence task. The model is trained to map a "scrambled" input expression (a product of transformed functions) to a "canonical" output expression (a reduced product of functions).
Dynamic Batching: To handle variable-length symbolic expressions efficiently, a dynamic batching algorithm is used. Instead of fixed batch sizes, sequences are grouped by token count to minimize padding overhead, reducing training time by approximately 30%.
Data Encoding: Mathematical expressions are parsed into syntax trees, serialized into prefix notation, and mapped to token IDs via a custom dictionary.

2. Data Generation and Scrambling

The training datasets are generated algorithmically by applying random sequences of known identities to canonical forms:

$q$ - $\theta$ Function: The dataset involves $SL(2, \mathbb{Z})$ transformations (generated by $S$ and $T$ matrices), shifts, reflections, inversions, and duplication identities. Scrambling is achieved by applying a sequence of $n_s$ elementary operators to a base expression, potentially introducing $n_t$ trivial pairs (identity insertions) to increase complexity.
Elliptic Gamma Function: The dataset involves $SL(3, \mathbb{Z})$ transformations, shifts in $\mathbb{Z}^3$ , inversions, and duplication identities. The complexity is higher due to the increased number of moduli ( $\tau, \sigma$ ) and the potential for a single transformation to split a function into a product of multiple functions.

3. Sampling and Modular Transformation Recognition (Section 3)

Before tackling the symbolic simplification, the authors investigate whether ML can learn to recognize $SL(2, \mathbb{Z})$ modular transformations on the upper half-plane.

Sampling Measures: Four different sampling measures on the Poincaré disk (mapped to the upper half-plane) were tested: Hyperbolic measure, and three variations of Bertrand's paradox (random chord endpoints, uniform area, random radial distance).
Goal: To map a point $z$ outside the fundamental domain to its image inside the fundamental domain and predict the corresponding $SL(2, \mathbb{Z})$ matrix.
Result: The model achieved high accuracy (up to 96.6% for hyperbolic measure), confirming that Transformers can learn the geometric structure of modular transformations.

4. Handling Modular Phases (Cocycles)

A critical distinction from polylogarithm simplification is the presence of non-trivial phase factors (modular cocycles) in elliptic function transformations.

Strategy: The ML model is trained only to identify the canonical representative (the simplified product of functions). It does not predict the phase factor directly.
Reconstruction: The modular cocycle (the phase difference between the scrambled and simplified forms) is reconstructed deterministically using a separate algorithm (detailed in Appendix A). This algorithm iteratively applies difference equations and known identities (e.g., relating Gamma ratios to $\theta$ functions) to compute the exact phase $e^{i\pi Q}$ .

Key Results

1. $q$ - $\theta$ Function Simplification

In-Distribution Performance: The model achieved 99.96% accuracy on test sets where the scrambling depth ( $n_s$ ) matched the training distribution ( $n_s \in [3, 5]$ ).
Extrapolation: When tested on deeper scrambling depths ( $n_s \in [6, 10]$ ), the model maintained 99.87% accuracy, demonstrating strong generalization capabilities.
Complex Transformations: When including all transformation types (including duplication), accuracy remained above 91%, though it decreased slightly with increasing formula length.

2. Elliptic Gamma Function Simplification

In-Distribution Performance: On a test set with $n_s \in [3, 6]$ , the model achieved 97.23% accuracy.
Extrapolation: For significantly deeper scrambling ( $n_s \in [7, 10]$ ), the model retained 93.02% accuracy.
Robustness: The model showed robustness to the number of identity insertions ( $n_t$ ). Interestingly, accuracy increased with $n_t$ (up to $n_t=3$ ), attributed to a dilution of the scrambling density, making the identity cancellations easier to detect.
Beam Search: Increasing the beam search size (from 1 to 10) provided negligible accuracy improvements (<2%) while significantly increasing inference time, suggesting that greedy or small-beam decoding is sufficient for this task.

3. Numerical Verification

The authors implemented a rigorous numerical verification protocol. Since the simplified expression differs from the input only by a phase factor (a polynomial in $z$ ), they verified predictions by:

Checking if the logarithmic ratio of the input and predicted expressions follows the expected polynomial dependence (quadratic for $\theta$ , cubic for $\Gamma$ ).
Using finite difference tests to confirm the vanishing of higher-order derivatives consistent with the expected polynomial degree.

Significance and Claims

The paper claims to demonstrate that modern sequence-to-sequence Transformer models can effectively identify canonical representatives of expressions related by modular identities. The significance lies in:

Automation of Symbolic Manipulation: The framework provides a practical tool for automating the simplification of expressions involving special functions that are central to supersymmetric QFT and string theory (e.g., partition functions on lens spaces $L(p,q) \times S^1$ ).
Generalization Beyond Training: The models exhibit robust extrapolation, successfully simplifying expressions with scrambling depths and structural complexities not seen during training. This suggests the models learn the underlying algebraic rules rather than merely memorizing patterns.
Modular Identity Discovery: By successfully mapping scrambled inputs to canonical forms, the framework implicitly learns the relevant modular groups ( $SL(2, \mathbb{Z})$ and $SL(3, \mathbb{Z})$ ) and their transformation laws.
Separation of Concerns: The approach of using ML for structural simplification and deterministic algorithms for phase reconstruction proves effective, avoiding the difficulty of learning complex transcendental phase functions directly.

The authors conclude that this framework offers a promising avenue for investigating modular structures in more complex settings, such as partition functions on general Seifert three-manifolds, where modular identities are less understood. They explicitly note that the current work focuses on specific classes of functions and that extending this to hybrid expressions (mixing polylogarithms, $\theta$ , and $\Gamma$ ) or higher-rank functions remains a future direction.