⚛️ lattice

Weight-Space Physics: Interpretable Hypernetworks for Lattice Quantum Field Theories

This paper introduces JEPAWG, a Joint-Embedding Predictive Architecture-based weight generator that maps coupling constants directly to normalizing flow weights, demonstrating that the network's latent space can recover critical physical phenomena like phase transitions and finite-size scaling, thereby establishing network weights themselves as interpretable physical observables for lattice field theories.

Original authors: Tobias Göbel, Julian R. Ebelt, Zier Mensch, Mathis Gerdes, Miranda C. N. Cheng

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tobias Göbel, Julian R. Ebelt, Zier Mensch, Mathis Gerdes, Miranda C. N. Cheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand a complex machine, like a high-end espresso maker. Usually, to get a perfect cup of coffee, you have to tweak the dials (temperature, pressure, grind size) and run thousands of test batches, tasting the result each time to see if you got it right. This is how physicists currently study the "strong nuclear force" (the glue holding atoms together). They have a set of rules (coupling constants), but predicting the outcome requires massive, slow computer simulations.

This paper proposes a new way to look at the machine itself. Instead of just tasting the coffee, they want to read the machine's internal wiring diagram to understand exactly how it works and predict what it will do next.

Here is the breakdown of their work using everyday analogies:

1. The Problem: The "Black Box" of Physics

Physicists use a method called Lattice Field Theory to simulate nature. Think of this as a giant grid of pixels representing space. To simulate reality, they set "dials" (coupling constants) on this grid.

The Old Way: They run a simulation, get a result, and if they want to know what happens with slightly different dials, they have to start the whole simulation over from scratch. It's slow and expensive.
The New Tool (Normalizing Flows): Recently, scientists built AI "samplers" (like a smart robot arm) that can quickly generate these results. But there's a catch: we don't know what the robot learned. It's a "black box." We know it works, but we can't read its brain.

2. The Big Idea: Reading the "Wiring Diagram"

The authors ask a simple question: Can we read the physics directly from the AI's internal settings (its weights)?

Imagine the AI's internal settings as a massive, intricate map. The authors believe this map isn't just random noise; it actually contains a hidden picture of the physical laws it was trained on. If you look at the shape of this map, you should be able to see where the "phase transitions" are (like water turning to ice) without ever running a single simulation.

3. The Solution: JEPAWG (The "Universal Translator")

To test this, they built a new AI tool called JEPAWG.

The Analogy: Imagine you have a library of books written in a strange language (the AI's internal weights). You want to know what the books are about, but you can't read the language.
How JEPAWG works: It acts like a translator that learns to connect the "dial settings" (the physics inputs) directly to the "book text" (the AI weights).
The Twist: Unlike other translators that just try to memorize the text, JEPAWG learns the structure of the language. It creates a "latent space"—a simplified, 2D map of the complex 3D world of the AI's brain.

4. What They Discovered

When they looked at the map JEPAWG created, they found something amazing. The map wasn't just a random blob; it perfectly mirrored the laws of physics.

Finding the Dimensions: The physics they were studying only has two main dials (temperature and pressure). Other AI methods tried to map this into a 3D or 4D space, making it messy and confusing. JEPAWG correctly figured out that the world only needs 2 dimensions. It "knew" the physics was simple, even though the AI's brain was huge and complex.
Spotting the Phase Transition: In physics, there's a "critical line" where things change drastically (like water boiling). The authors found a "ridge" or a bump in the AI's map that lined up perfectly with this critical line. The AI had learned to highlight the most important part of the physics just by organizing its own internal weights.
The "Size" Trick (Renormalization Group): This is the most magical part. The authors tested the AI on grids of different sizes (small vs. large). In physics, a small grid and a large grid can represent the same reality if you scale them correctly.
- JEPAWG learned this scaling rule automatically. It realized that a "small" theory and a "large" theory are actually neighbors on its map if they represent the same physics.
- It even calculated the correct "scaling exponent" (a number that describes how things grow), matching the famous value of the 2D Ising model (a standard physics benchmark). It did this without being told the answer; it just figured it out by looking at the shape of the map.

5. Why This Matters

It's a Generator: Because JEPAWG understands the relationship between the dials and the AI's brain, it can predict what the AI's brain should look like for a new dial setting it has never seen before. It's like looking at a blueprint and instantly knowing how to build a new version of the machine without having to build it first.
It's Robust: Even when the training data was messy or came from different sources (like two different robots building the same thing), JEPAWG still managed to create a smooth, continuous map. Other methods (like standard Autoencoders) got confused and created disjointed, broken maps.

Summary

The paper claims that neural network weights are not just numbers; they are a new kind of physical observable. By using a special architecture called JEPAWG, they showed that you can "read" the laws of physics directly from the geometry of an AI's brain. The AI didn't just learn to simulate the physics; it learned to organize itself in a way that perfectly reflects the underlying reality, allowing scientists to predict outcomes and understand phase transitions just by looking at the network's structure.

Technical Summary: Weight-Space Physics: Interpretable Hypernetworks for Lattice Quantum Field Theories

Problem Statement
Lattice Field Theory (LFT) is the primary tool for simulating non-perturbative phenomena in high-energy and condensed matter physics. While the underlying physics is parametrized analytically by bare coupling constants ( $g$ ), extracting physical observables typically requires extensive numerical simulation because these bare parameters are weak predictors of the resulting Boltzmann distributions. Recently, flow-based generative models (normalizing flows) have emerged as effective samplers for these distributions, learning an invertible map from a base distribution to the target Boltzmann measure. However, a critical gap remains: while these networks learn to sample, it is difficult to interpret what physical structure they have encoded in their weights ( $\theta_g$ ). Specifically, it is unclear if the geometry of the weight space recovers the physical structure obscured by bare couplings, and whether weights for unseen theories can be generated without retraining.

Methodology: JEPAWG
The authors introduce JEPAWG (Joint-Embedding Predictive Architecture-based Weight Generator), a framework designed to map coupling constants directly to flow network weights via a learned latent space. Unlike standard hypernetworks that regress couplings to weights using reconstruction loss, JEPAWG utilizes a JEPA architecture (LeCun, 2022; Assran et al., 2023) to learn representations by prediction rather than reconstruction.

The architecture consists of four components:

Weight Encoder ( $E_\theta$ ): Maps canonicalized flow parameters $\theta$ to a latent embedding $z_\theta$ .
Coupling Encoder ( $E_g$ ): Maps bare couplings $g = (m^2, \lambda)$ to a latent embedding $z_g$ .
Predictor ( $P$ ): Maps the coupling embedding $z_g$ to the predicted weight embedding $\hat{z}_\theta$ .
Decoder ( $D_\theta$ ): Maps the predicted embedding back to the flow parameters $\hat{\theta}$ .

The model is trained using a modified VICReg objective (Bardes et al., 2022), which enforces invariance (alignment between predicted and target embeddings), variance (preventing representation collapse), and explicitly removes the covariance term to allow the latent space to concentrate on the low-dimensional structure of the problem. A crucial preprocessing step involves weight canonicalization, which resolves symmetries (basis changes, permutations, sign flips) in the weight space to ensure a unique representative for each symmetry orbit, thereby facilitating non-ambiguous feature extraction.

The system is applied to the two-dimensional real scalar $\phi^4$ theory on periodic square lattices ( $L \in \{6, \dots, 11\}$ ). The training data is constructed from two independently trained conditional flows (multi-seed setting) covering disjoint regions of the coupling space to ensure the learned representations are robust to structural discontinuities in the weight manifold.

Key Contributions and Results
The paper presents four primary contributions, demonstrating that the geometry of network weights can serve as a new type of physical observable:

Recovery of Physical Dimensionality:
The JEPAWG latent space successfully recovers the intrinsic dimension of the underlying 2D coupling manifold. Using Maximum Likelihood Estimation (MLE), the latent dimension is estimated at $2.01 \pm 0.06$ , with 98.5% of variance captured by two principal axes. In contrast, baseline methods (Autoencoders, VAEs, and PCA) inflate this dimension significantly (e.g., VAE MLE $\approx 4.88$ ), failing to capture the low-dimensional physical structure.
Phase Transition Detection from Weights:
By computing the pullback metric on the parameter map $\theta(g)$ , the authors identify a "ridge" in the weight space that tracks the critical line of the $\phi^4$ theory. Crucially, this signal originates exclusively from non-local components of the network (zero-mode scaling and Fourier scaling parameters), while local "bulk" groups show no correlation with criticality. This allows the phase boundary to be located without drawing any physical samples.
Renormalization Group (RG) Structure and Finite-Size Scaling:
The latent geometry organizes theories according to the finite-size scaling (FSS) variable. When comparing weights across different lattice sizes ( $L$ ), the latent distance is minimized when the scaling exponent $\nu$ aligns with the theoretical 2D Ising value ( $\nu = 1$ ). The learned exponent $\nu^*$ brackets the exact value (e.g., $1.00^{+0.08}_{-0.06}$ for the $6 \to 11$ pairing), indicating the network has implicitly learned RG-invariant combinations rather than just the bare couplings.
Robust Generation:
As a generator, JEPAWG effectively interpolates and extrapolates to unseen couplings. In the challenging multi-seed setting, JEPAWG achieves an Effective Sample Size (ESS) of 0.88 for interpolation and maintains robust performance on edges (0.56) and corners (0.39). Baselines (PCA, AE, VAE) collapse significantly in this setting (ESS $\approx 0.30\text{--}0.40$ ), largely because they fail to handle the discontinuities introduced by the multi-seed training data and weight canonicalization.

Significance and Claims
The paper claims that physics acts as a supplier of natural synthetic data that is rich enough to be non-trivial yet structured enough to be interpretable, providing a ground truth often lacking in general machine learning interpretability research.

The central finding is that the geometry of trained weights reflects the geometry of the physical theory, concentrating specifically on the network components responsible for non-local transformations where the physics "lives." The authors argue that treating network weights as physical observables allows for the direct reading of physical structure (dimensionality, phase transitions, and scaling exponents) without supervision from physical labels. This validates the JEPAWG framework and supports the broader claim that physics-based synthetic data is effective for developing interpretable AI.

Limitations
The authors note that their experiments are confined to a single 2D theory on small lattices ( $L \le 11$ ) due to computational costs. Consequently, their interpretability claims rest on agreement with a single universality class (2D Ising). They do not claim the method works for all quantum field theories or that it provides high-precision measurements of critical exponents, but rather a qualitative test of RG-awareness in the latent space. Future work is proposed to extend this to larger lattices and diverse physical systems.