Neural Langevin Machine: a local asymmetric learning rule can be creative

Technical Summary: Neural Langevin Machine

Problem Statement
Current artificial neural network-based generative models, such as Variational Autoencoders, Generative Adversarial Networks, and Diffusion Models, face significant challenges regarding biological plausibility and training complexity. Specifically, they often lack:

1. Asymmetric Recurrence: Biological neural networks possess asymmetric synaptic couplings, whereas many popular models rely on symmetric structures or require global error signals.
2. Local Learning: Biological circuits learn via local signals, while most generative models depend on global error backpropagation.
3. Utilization of Neural Dynamics: While neural response variability and fixed points (or chaotic attractors) can store and generate examples, the potential of neural response variability in generative sampling is under-explored.
4. Thermodynamic Reversibility: Previous attempts to use Langevin dynamics often enforce thermodynamic reversibility, which violates biological constraints and fails to model the brain's perception and imagination processes effectively.

Methodology
The authors propose the Neural Langevin Machine (NLM), a generative model that leverages the fixed points of recurrent neural networks (RNNs) with random asymmetric couplings.

• Dynamics: The model is defined by a stochastic differential equation (Langevin dynamics):
  $$\frac{dx}{dt} = -\nabla_x E(x) + \sqrt{2T}\epsilon(t)$$
  where $x$ is the synaptic current vector, $T$ is a temperature parameter, and $\epsilon(t)$ is Gaussian white noise.
• Energy Function: The potential energy $E(x)$ is constructed to stabilize the exponentially many unstable fixed points inherent in chaotic RNNs. It includes an Onsager feedback term derived from the gradient of the energy, which stabilizes chaotic fluctuations. The energy function takes the form:
  $$E(x) = \frac{1}{2} \sum_{i} \left( -x_i + b_i + \sum_{j} J_{ij}\phi(x_j) \right)^2$$
  where $\phi(x) = \tanh(x)$ is the firing rate function, $b_i$ is the bias, and $J_{ij}$ represents asymmetric neural coupling.
• Learning Rule: The model optimizes coupling parameters $\{J_{kl}\}$ to minimize the Kullback-Leibler (KL) divergence between the model distribution and the data distribution. This minimization follows a minimum entropy principle, leading to a local, asymmetric learning rule:
  $$\frac{\partial L}{\partial J_{kl}} = \beta \left[ \langle (x_k - h_k)\phi(x_l) \rangle_{\text{model}} - \langle (x_k - h_k)\phi(x_l) \rangle_{\text{data}} \right]$$
  Here, $h_k$ is the integrated synaptic current. The term $(x_k - h_k)$ represents a postsynaptic contribution interpreted as prediction error (reducing the speed of postsynaptic activity). This rule is local, requiring only neural signals, and is asymmetric with respect to neuron indices.
• Training Protocols: The authors implement several training strategies to explore out-of-equilibrium (OOE) regimes:
  • PCD-k (Persistent Contrastive Divergence): Uses persistent chains updated for $k$ steps between parameter updates.
  • Rdm-k: Re-initializes Langevin chains from random configurations at every gradient estimation step.
  • AE-NLM: Trains an autoencoder to compress data into a latent space, where the NLM is trained, significantly reducing computational complexity.

Key Contributions

1. Biologically Plausible Generative Model: The NLM satisfies three criteria: local learning, neural-dynamics-based sampling, and asymmetric recurrence. It derives a learning rule interpretable as speed-adjusted predictive learning.
2. Stabilization of Unstable Fixed Points: The work demonstrates that the Onsager feedback term in the energy function stabilizes the unstable fixed points of chaotic RNNs, making them usable for memory and generation.
3. Local Asymmetric Learning Rule: The derived learning rule connects generative modeling to fundamental physical principles (minimum entropy) and aligns with the "Bayesian brain without probability" hypothesis and Yann LeCun's energy-based AI path.
4. Out-of-Equilibrium (OOE) Regime: The paper identifies an OOE regime where the best generative performance is achieved without waiting for the system to reach thermodynamic equilibrium, mimicking continuous brain dynamics driven by stochastic noise.

Results

• Generative Performance: Applied to MNIST (handwritten digits) and Fashion MNIST, the NLM successfully generates samples. Visualizations show continuous state transitions between different digit classes (e.g., from '6' to '1'), driven by stochastic noise.
• Memorization-to-Generalization Transition: As the training data size increases, the model transitions from pure memorization (overfitting) to generalization. This transition is quantified using an Adversarial Accuracy Indicator (AAI).
  • In data-deficient regimes, the model memorizes training samples.
  • Beyond a specific data threshold, the model generates samples statistically indistinguishable from real data.
  • Excessive data size can eventually degrade quality, suggesting a saturation point.
• Eigenvalue Spectrum: During learning, the spectrum of the coupling matrix $J$ evolves from isotropic to an "anchor" shape, with eigenvalues distributed around the positive real axis playing a key role in generalization.
• Denoising and Associative Memory: The NLM functions as an associative memory network. When presented with corrupted images (25% pixel noise), the dynamics converge to clean digit representations, demonstrating a new type of associative memory with asymmetric coupling.
• Latent Space Enhancement: Using the AE-NLM protocol on Fashion MNIST (10 categories) improves generation quality and allows for sampling in a lower-dimensional latent space.

Significance and Claims
The authors claim that the NLM provides a promising avenue for neuro-inspired generative models that allow for the dynamic exploration of the "creative activity space of imagination." By utilizing unstable fixed points and local asymmetric learning, the model bridges the gap between biological neural mechanisms and engineering objectives. The work suggests that the brain may utilize similar mechanisms for perception, memory, and imagination, where continuous exploration of phase space facilitates transitions from memory to generalization. The paper positions this framework as a step toward understanding subjective perception and imagination in the brain, while also offering a computationally efficient alternative to current dominant generative models through latent space training.