🔬 condensed matter

Learning and Testing Inverse Statistical Problems For Interacting Systems Undergoing Phase Transition

This paper provides a theoretical overview and practical application of Maximum Likelihood, Maximum Pseudo-Likelihood, and Mean-Field methods for solving inverse statistical problems in various interacting systems undergoing phase transitions, accompanied by a reproducible GitHub repository.

Original authors: Stefano Bae, Dario Bocchi, Luca Maria Del Bono, Luca Leuzzi

Published 2026-06-30

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Stefano Bae, Dario Bocchi, Luca Maria Del Bono, Luca Leuzzi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Solving a Puzzle Backwards

Imagine you walk into a room and see a complex machine running. You can see the gears turning, the lights flashing, and the sounds it makes (the data). But you don't have the blueprint, and you don't know how the gears are connected or what the settings on the dials are (the model).

In physics, this is called an Inverse Problem. Usually, scientists do the opposite: they have the blueprint, they set the dials, and they predict what the machine will do. This paper asks: What if we only have the machine running, and we need to figure out the blueprint just by watching it?

The authors focus on a specific, tricky kind of machine: one that undergoes a Phase Transition. Think of this like water. At high heat, it's a chaotic gas (spins are random). At low heat, it freezes into a solid block (spins line up). The behavior changes drastically depending on the temperature. The paper tests if we can still figure out the machine's settings when it's behaving like a gas, a solid, or right in the middle of freezing.

The Three Tools in the Toolbox

To solve this puzzle, the authors test three different methods. They compare how well each method can guess the hidden connections between the parts of the machine.

1. The "Perfect Detective" (Maximum Likelihood)

Imagine you are a detective trying to guess the rules of a game by watching thousands of games being played.

How it works: You try to find the set of rules that makes the games you watched the most likely to happen.
The Catch: To do this perfectly, you have to calculate the probability of every single possible outcome the machine could produce. If the machine has 100 parts, the number of possibilities is so huge (like the number of atoms in the universe) that it takes forever to calculate. It's like trying to count every grain of sand on a beach to find the one that is slightly different.
Verdict: It's the most accurate in theory, but it's too slow and expensive to use for big systems.

2. The "Smart Shortcut" (Maximum Pseudo-Likelihood)

Since the "Perfect Detective" is too slow, the authors use a clever shortcut.

How it works: Instead of looking at the whole machine at once, the detective looks at one gear at a time. They ask: "If I freeze all the other gears in the positions I saw them, what is the most likely setting for this specific gear?"
The Analogy: Imagine trying to guess the recipe for a soup. Instead of tasting the whole pot and guessing every ingredient at once, you taste one spoonful, assume the other ingredients are fixed, and guess the salt. Then you move to the next spoonful.
The Result: This method is much faster. The paper shows that even though it's a shortcut, it works almost as well as the perfect method, especially when the machine is near a phase transition (like the freezing point).

3. The "Rule of Thumb" (Mean-Field)

This is the simplest approach, like using a rough estimate based on averages.

How it works: It assumes that every part of the machine is influenced by the average behavior of all the other parts, rather than specific neighbors. It ignores the messy, specific details and looks at the big picture.
The Analogy: If you want to know the weather in a city, you don't measure every single street corner. You just look at the average temperature reported by the main station.
The Result: It's very fast and easy to calculate. However, it's less accurate than the "Smart Shortcut," especially when the machine is in a complex state (like a spin-glass, where parts are fighting each other).

The Experiments: Testing the Tools

The authors didn't just talk about theory; they built a virtual lab. They created four different types of "machines" (mathematical models) and ran simulations to generate data:

The Ordered Machine (Ising): A neat grid where everything wants to line up (like a magnet).
The Messy Machine (Spin Glass): A grid where some parts want to line up and others want to fight, creating a chaotic mess.
The Multi-Color Machine (Potts): Instead of just "Up" or "Down," parts can point in four different directions.
The "Hole" Machine (Blume-Capel): A machine where parts can be "Up," "Down," or "Empty" (like a hole in the system).

They tested these machines on two shapes: a neat grid (like a chessboard) and a random web (like a social network where connections are random).

What They Found

Temperature Matters: The "Smart Shortcut" (Pseudo-Likelihood) worked best when the data was collected at a "Goldilocks" temperature—not too hot (too random) and not too cold (too frozen). If the machine is too cold, it gets stuck in one state, and the detective can't see enough variety to guess the rules.
The Shortcut Wins: The "Smart Shortcut" consistently outperformed the "Rule of Thumb" (Mean-Field). It was better at guessing the connections, even in the messy, chaotic machines.
Cleaning Up the Noise: They also tested a technique called "Lasso." Imagine you are trying to guess the recipe, but you suspect many ingredients aren't actually in the soup. Lasso helps you ignore the ingredients that don't matter, making the final guess cleaner and more accurate, especially for the neat, ordered machines.

The Takeaway

The paper is a guidebook for scientists who want to reverse-engineer complex systems. It shows that you don't need to be a supercomputer to solve these puzzles. By using the "Smart Shortcut" (Pseudo-Likelihood) and being careful about the temperature at which you collect your data, you can accurately figure out how a complex system is connected, even when that system is undergoing a dramatic change in behavior.

The authors also made all their code available online, inviting others to play with the data and try these methods on their own systems.

Technical Summary: Learning and Testing Inverse Statistical Problems for Interacting Systems Undergoing Phase Transition

Problem Statement
The paper addresses the inverse problem in statistical physics: inferring the underlying model parameters (specifically interaction couplings $J_{ij}$ and external fields $h_i$ ) from observed data (spin configurations), rather than the standard "direct" problem of predicting observables from known parameters. The central challenge investigated is how these inference methods perform when the system undergoes phase transitions. In such regimes, the nature of the data changes qualitatively (e.g., from disordered paramagnetic to ordered ferromagnetic or spin-glass states), potentially affecting the stability and accuracy of reconstruction algorithms. The authors aim to provide a pedagogical and practical framework for solving these inverse problems across various statistical mechanics models, including those with first-order and second-order transitions, on both regular lattices and random graphs.

Methodology
The authors evaluate three primary inference methodologies, deriving them from equilibrium statistical mechanics and contrasting them with machine learning approaches:

Maximum Likelihood (ML): Theoretically optimal, this method maximizes the posterior probability of the parameters given the data. It involves minimizing the difference between empirical averages (magnetization and correlations) and theoretical averages derived from the Boltzmann-Gibbs distribution. However, the authors note that exact ML requires computing the partition function $Z$ , which involves summing over all $2^N$ (or $q^N$ ) configurations, rendering it computationally infeasible for large systems without approximations.
Maximum Pseudo-Likelihood (PL): To circumvent the intractability of the partition function, the PL method approximates the joint probability by the product of conditional probabilities for each spin given all others. This transforms the inference problem into a series of logistic regression problems (or multinomial logistic regression for Potts models). The method relies on the assumption that the measured data samples are statistically representative of the equilibrium ensemble. The authors implement iterative gradient descent updates for the parameters, comparing empirical correlations with those predicted by the conditional model.
Mean-Field (MF) Approach: This method utilizes the naive mean-field approximation, which assumes factorized joint distributions and vanishing connected correlations in the thermodynamic limit. It provides a direct analytical solution where the coupling matrix is inferred as the negative inverse of the empirical covariance matrix ( $\beta J_{ij} = -(\Gamma^{-1})_{ij}$ ). While exact for infinite-dimensional or random graph models, it serves as an approximation for low-dimensional lattices.

The study also incorporates regularization (specifically $\ell_1$ -Lasso and $\ell_2$ norms) to prevent overfitting, particularly when the number of parameters exceeds the amount of data or when inferring sparse networks.

Models and Data Generation
The methods are tested on synthetic data generated via Markov Chain Monte Carlo (MCMC) simulations (using Parallel Tempering/Exchange Monte Carlo and Wolff algorithms) for four distinct systems:

Ising Model: Both ferromagnetic (ordered couplings) and spin-glass (disordered couplings) variants on 2D square lattices and Erdős–Rényi (ER) random graphs.
Vector Potts (Clock) Model: A 4-state model on 3D cubic lattices and ER graphs.
Blume-Capel Model: A spin-1 model with single-ion anisotropy on 2D lattices and ER graphs, capable of exhibiting both first-order and second-order phase transitions, as well as a tricritical point.

Key Results

Performance Comparison: The Maximum Pseudo-Likelihood (PL) method consistently performs equally to or better than the Mean-Field (MF) approach across all temperatures and models.
Role of Regularization: In ferromagnetic systems with ordered couplings, adding Lasso ( $\ell_1$ ) regularization significantly reduces reconstruction error by enforcing sparsity. In disordered systems (spin-glasses), the benefit of Lasso is less consistent, sometimes improving results at specific temperatures but degrading them at others.
Temperature Dependence: Reconstruction accuracy generally degrades near critical temperatures ( $T_c$ ) due to critical slowing down and large fluctuations, though the PL method remains robust.
First-Order Transitions: In the Blume-Capel model, the authors observe that inference becomes particularly difficult in regions dominated by first-order transitions or where the system is in a paramagnetic phase driven by "holes" (spin $s_i=0$ ). In these regimes, the lack of correlation information from zero spins leads to high reconstruction errors, even for the PL method.
Ranking Analysis: The authors demonstrate that while absolute coupling values may have errors, the ranking of couplings by magnitude is preserved well, especially with larger datasets. This allows for the correct identification of the network topology (i.e., which links exist) even if the precise coupling strengths are slightly off.

Significance and Claims
The paper positions itself as a pedagogical bridge between statistical mechanics and machine learning, offering a "toolbox" for inverse problems in the presence of phase transitions. Its primary contributions are:

Pedagogical Clarity: It provides a structured introduction to ML, PL, and MF methods, explicitly distinguishing between statistical inference (focus on model parameters and uncertainty) and machine learning (focus on predictive accuracy), while acknowledging their blurred boundaries.
Systematic Benchmarking: It offers a comprehensive comparison of these methods across models with different transition orders (first vs. second) and topologies (lattice vs. random graph), highlighting specific failure modes (e.g., in first-order transition regions).
Reproducibility: The authors emphasize the practical utility of their work by providing a GitHub repository containing code for data generation (MCMC) and inference, allowing users to reproduce results and experiment with new systems.

The authors modestly claim that their work serves as both a theoretical introduction and a practical tool for hands-on learning, enabling the reconstruction of interaction matrices in complex systems where data nature changes due to phase transitions. They do not claim to solve all inverse problems but rather to illustrate the efficacy and limitations of standard statistical mechanics techniques in this context.