The solution of the Hopfield model with Gaussian patterns

Technical Summary: The Solution of the Hopfield Model with Gaussian Patterns

Problem Statement
The paper addresses the rigorous derivation of the thermodynamic limit for the free energy (quenched pressure) of the Hopfield model with Gaussian patterns. In this model, $N$ Ising spins interact via $M = \alpha N$ stored patterns $\xi^\mu \in \mathbb{R}^N$. While the replica theory of Amit, Gutfreund, and Sompolinsky (AGS) predicts a phase diagram including paramagnetic, spin-glass, and retrieval phases, rigorous mathematical results have historically been limited to the replica-symmetric regime (small load $\alpha$ or high temperature $T$) or the annealed region. A fundamental obstacle has been the non-convexity of the Hopfield Hamiltonian's Hubbard–Stratonovich representation, which describes a bipartite system between visible Ising spins and continuous Gaussian hidden units. Standard interpolation techniques (Guerra–Toninelli) and multi-species theories rely on convexity conditions that fail for the product of overlaps characteristic of this bipartite structure. Furthermore, the existence of the thermodynamic limit outside these restricted regimes had not been established.

Methodology
The authors derive the large $N$ limit of the quenched pressure through a three-step mechanism:

1. Hubbard–Stratonovich Transformation: The model is mapped onto a bipartite system consisting of $N$ Ising spins and $M$ Gaussian hidden units. The interaction is linear in the Gaussian variables.
2. Radial Decomposition and Laplace Principle: The authors observe that a Gaussian vector in $\mathbb{R}^M$ can be decomposed into a random radius $s$ and a uniform direction $\tau$ on a sphere. The radius $s$ acts as a thermodynamic variable with an entropic cost $I_\alpha(s)$ derived from the distribution of the Gaussian norm. The partition function is expressed as an integral over the radius $s$ of the partition function of an Ising–spherical model with effective coupling $\gamma = \beta s^2$. By applying the Laplace principle, the free energy is shown to converge to a variational principle involving a supremum over the radius $s$.
3. Solution of the Mixed Model: The resulting Ising–spherical model (with Gaussian couplings) is solved using the Hamilton–Jacobi approach developed by Mourrat. This method treats the free energy as a function of a fictitious time (coupling strength) and Parisi order parameters. Because the covariance of the bipartite model is not convex, the solution is not the standard Parisi variational principle but the Hopf formula, which involves a supremum over conjugate variables (overlaps) and an infimum over the Parisi order parameters.

Key Contributions and Results

• Existence of the Thermodynamic Limit: The paper proves that the free energy converges almost surely and in expectation to a well-defined variational principle for every load $\alpha > 0$ and temperature $T > 0$.
• Explicit Variational Principle: The limit pressure $P(\alpha, \beta)$ is given by:
  $$P(\alpha, \beta) = \sup_{s>0} \left\{ \phi_\alpha(\beta s^2) - I_\alpha(s) \right\}$$
  where $\phi_\alpha$ is the pressure of the mixed Ising–spherical model and $I_\alpha(s)$ is the relative entropy cost of the radius.
• Thermodynamic Variable (Radius): The radius of the hidden layer (empirical variance of the hidden units) is identified as a thermodynamic variable. The paper derives an equation of state for this radius, showing how it balances the entropic preference for $s^2=1$ against the energetic drive for larger fluctuations.
• Rigorous Phase Diagram:
  • Annealed Region: The model is annealed (free energy equals annealed free energy) for $T \ge T_g = 1 + \sqrt{\alpha}$.
  • Glassy Region: For $T < T_g$, the pressure is strictly less than the annealed value. The transition at $T_g$ is identified as the rigorous boundary of the annealed region.
  • No Retrieval Phase: For Gaussian patterns at positive load, the paper confirms (via symmetry arguments) the absence of a retrieval phase in the symmetric Gibbs measure.
• Replica Symmetry and AGS Theory:
  • Restricting the variational principle to constant paths recovers the AGS replica-symmetric functional as a rigorous upper bound.
  • The paper proves that the AGS spin-glass solution is unique.
  • Replica Symmetry Breaking (RSB): Below $T_g$, the replica-symmetric solution is unstable. The instability condition reduces exactly to the zero-field de Almeida–Thouless (AT) condition of the Sherrington–Kirkpatrick (SK) model. Consequently, the entire spin-glass phase is replica-symmetry-broken, and the AGS free energy is a strict upper bound in this region.

Significance
The paper provides the first assumption-free derivation of the AGS functional as an upper bound valid at all temperatures. It resolves the long-standing issue of the non-convexity in the Hopfield model by introducing the radius of the hidden layer as a thermodynamic variable and utilizing the Hamilton–Jacobi/Hopf formula framework. This approach successfully extends rigorous results to the full phase diagram, including the low-temperature spin-glass phase where replica symmetry is broken. The work establishes that the instability of the AGS solution in the Hopfield model is structurally equivalent to the AT instability in the SK model, providing a complete and rigorous characterization of the model's thermodynamics for Gaussian patterns.