Technical Summary: Finite-Nudge Equilibrium Propagation in Thermal Ensembles

Problem Statement

Equilibrium Propagation (EP) and Contrastive Hebbian Learning (CHL) offer biologically plausible alternatives to backpropagation by utilizing local learning rules based on energy minimization. However, classical formulations of these methods rely on two restrictive assumptions that limit their theoretical and practical applicability:

1. The Infinitesimal Limit: Traditional EP requires the "nudge" (the supervisory signal applied to the output) to be infinitesimally small ($\beta \to 0$) to recover the exact gradient of the supervised loss. In practice, finite nudging is often required for stable learning, but this introduces bias relative to the true gradient.
2. Deterministic Dynamics: Classical analyses often assume zero-temperature dynamics where the network state collapses to a single deterministic energy minimum. This idealization fails to account for the stochastic nature of physical or biological circuits and struggles with non-convex energy landscapes where multiple local minima exist.

Consequently, the theoretical status of finite-nudge learning in stochastic, thermal ensembles has remained incomplete, leaving open the question of what global objective such learning rules actually optimize.

Methodology

The paper reframes the network state not as a deterministic point but as a random variable distributed according to a Gibbs-Boltzmann distribution at finite temperature $T$. The system is defined by an energy function $E(\theta, s)$ and a task-dependent loss $\ell(s)$.

The Stochastic Contrastive Objective

The author defines a new objective function, the Stochastic Contrastive Objective $J(\theta)$, as the difference in Helmholtz free energy between two phases:

• Free Phase ($\beta=0$): The network relaxes under the energy function $E(\theta, s)$ alone.
• Nudged Phase ($\beta=1$): The network relaxes under the objective kernel $F(\theta, \beta, s) = E(\theta, s) + \beta\ell(s)$.

The objective is defined as:
$$J(\theta) \triangleq A(\theta, 1) - A(\theta, 0)$$
where $A(\theta, \beta) = -T \log Z_\beta(\theta)$ is the Helmholtz free energy and $Z_\beta$ is the partition function.

Gradient Derivations

The paper establishes two exact representations for the gradient $\nabla_\theta J(\theta)$ without assuming infinitesimal nudging or convexity:

1. Expectation Contrast (Theorem 3.1):
   The gradient is exactly the difference between the expected local energy derivatives under the nudged and free Gibbs distributions:
   $$\nabla_\theta J(\theta) = \mathbb{E}_{s \sim \rho_1}[\nabla_\theta E(\theta, s)] - \mathbb{E}_{s \sim \rho_0}[\nabla_\theta E(\theta, s)]$$
   This proves that the standard two-phase contrastive update rule performs exact gradient descent on $J(\theta)$ for any finite temperature and finite nudging strength.

2. Integrated Covariance (Theorem 3.4):
   The gradient can also be expressed as an integral over the nudging strength $\beta \in [0, 1]$ of the covariance between the loss and the energy gradient:
   $$\nabla_\theta J(\theta) = -\frac{1}{T} \int_0^1 \text{Cov}_{s \sim \rho_\beta}[\ell(s), \nabla_\theta E(\theta, s)] \, d\beta$$
   This formulation generalizes the infinitesimal EP rule (which approximates this integral via a first-order Taylor expansion at $\beta=0$) to arbitrary finite nudging.

Zero-Temperature and Variational Limits

• Zero-Temperature Limit: As $T \to 0$, the theory recovers the classical deterministic contrastive rule. If the energy landscape has multiple minima, the gradient converges to a curvature-weighted average of the gradients at those minima (Laplace approximation), rather than a uniform average or a single arbitrary minimum.
• Variational Interpretation: Using the Gibbs variational principle, the objective $J(\theta)$ is shown to be a regularized proxy for the supervised loss. It decomposes into the expected loss in the nudged phase plus a Kullback-Leibler (KL) divergence term penalizing the distance between the nudged and free distributions:
  $$J(\theta) = \mathbb{E}_{s \sim \rho_1}[\ell(s)] + T \cdot \text{KL}(\rho_1 \parallel \rho_0)$$

Key Results

The author validates their theoretical framework through numerical experiments on the Fashion-MNIST dataset using a single-layer energy-based network with tanh units.

1. Training Performance:

   • Infinitesimal EP ($\beta=0.01$): Failed to learn, stalling near chance accuracy (20–30%).
   • Finite-Nudge EP ($\beta=1.0$): Rapidly achieved $\sim80\%$ test accuracy, closely tracking standard backpropagation.
   • Path-Integral EP: A variant implementing the integral covariance rule also achieved high accuracy.

2. Signal-to-Noise Ratio (SNR):

   • For small nudging strengths ($\beta \lesssim 10^{-2}$), the update signal was indistinguishable from sampling noise.
   • As $\beta$ approached 1.0, the SNR improved by an order of magnitude, demonstrating that finite nudging unlocks a signal regime inaccessible to infinitesimal methods.

3. Gradient Alignment:

   • The cosine similarity between the practical contrastive update and the true free-energy gradient increased monotonically with $\beta$, reaching $\sim0.5$ at $\beta=1.0$. This confirms that large nudges remain gradient-like with respect to the thermodynamic objective.

4. Exact Verification:

   • Additional experiments on a smaller dataset (Digits) with exact mathematical marginalization (avoiding MCMC noise) confirmed that the gradient identity holds exactly at finite $\beta$ and that the relative error between analytical and numerical derivatives remains at machine precision.

Significance and Claims

The paper claims to resolve the theoretical incompleteness of contrastive learning in stochastic settings. Its primary contributions are:

• Decoupling from the Infinitesimal Limit: The work proves that finite-nudge learning is not merely a biased approximation of backpropagation but is exact gradient descent on a well-defined Helmholtz free energy difference.
• Biological Plausibility: By validating the classic two-phase contrastive update as an exact gradient estimator for arbitrary finite nudging, the paper strengthens the case for local learning rules that do not require weight transport or infinitesimal perturbations, which are difficult to reconcile with biological constraints.
• Handling Non-Convexity and Stochasticity: The framework provides a rigorous foundation for learning in non-convex energy landscapes at finite temperatures, showing that the learning rule remains well-defined even when global minima are not unique or when the system operates in a thermal regime.
• Practical Advantage: The results suggest that the "small-nudge" approximation, often used in theoretical analyses, is practically detrimental due to noise. Finite nudging provides a necessary signal-to-noise advantage for effective training.

The author positions this work as a statistical mechanics foundation that unifies EP, CHL, and variational inference, linking the mechanics of local learning to the minimization of free energy and information-theoretic costs.